* Recovery/Access of imsm raid via mdadm?
@ 2013-01-10 16:23 chris
2013-01-10 17:09 ` Dave Jiang
0 siblings, 1 reply; 18+ messages in thread
From: chris @ 2013-01-10 16:23 UTC (permalink / raw)
To: Linux-RAID
Hello,
I have a machine which was running a imsm raid volume, where the
motherboard failed and I do not have access to another system with
imsm. I remember noticing some time ago that mdadm could recognize
these arrays, so I decided to try recovery in a spare machine with the
disks from the array.
I guess my questions are:
Is this the right forum for help with this?
Am I even going down a feasible path here or is this array dependent
on the HBA in some way?
If it is possible any ideas of anything else I can do to debug this further?
The original array was a raid 5 of 4x2TB sata disks
When I examine the first disk, things look good:
mdadm --examine /dev/sdb
mdadm: /dev/sdb is not attached to Intel(R) RAID controller.
mdadm: /dev/sdb is not attached to Intel(R) RAID controller.
/dev/sdb:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.3.00
Orig Family : 226cc5df
Family : 226cc5df
Generation : 000019dc
Attributes : All supported
UUID : b67ea029:aaea7da2:2540c1eb:ebe98af1
Checksum : 651263bf correct
MPB Sectors : 2
Disks : 4
RAID Devices : 1
Disk02 Serial : Z1E1RPA9
State : active
Id : 00030000
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
[Volume0]:
UUID : 51a415ba:dc9c8cd7:5b3ea8de:465b4630
RAID Level : 5
Members : 4
Slots : [__U_]
Failed disk : 0
This Slot : 2
Array Size : 11721072640 (5589.04 GiB 6001.19 GB)
Per Dev Size : 3907024648 (1863.01 GiB 2000.40 GB)
Sector Offset : 0
Num Stripes : 15261814
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : failed
Dirty State : clean
Disk00 Serial : Z1E1AKPH:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
Disk01 Serial : Z24091Q5:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
Disk03 Serial : Z1E19E4K:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
When I try to scan for arrays I get this:
# mdadm --examine --scan
HBAs of devices does not match (null) != (null)
ARRAY metadata=imsm UUID=b67ea029:aaea7da2:2540c1eb:ebe98af1
ARRAY /dev/md/Volume0 container=b67ea029:aaea7da2:2540c1eb:ebe98af1
member=0 UUID=51a415ba:dc9c8cd7:5b3ea8de:465b4630
ARRAY metadata=imsm UUID=b67ea029:aaea7da2:2540c1eb:ebe98af1
ARRAY /dev/md/Volume0 container=b67ea029:aaea7da2:2540c1eb:ebe98af1
member=0 UUID=51a415ba:dc9c8cd7:5b3ea8de:465b4630
My first concern is the warning that the HBA is missing, the whole
reason I am going at it this way is because I don't have the HBA.
Second concern is duplicate detection of the same array.
If i try to run # mdadm -As:
mdadm: No arrays found in config file or automatically
I also tried adding the output from --examine --scan to
/etc/mdadm/mdadm.conf but after trying that I now get blank output:
# mdadm --assemble /dev/md/Volume0
#
# mdadm --assemble --scan
#
full examine of all disks involved:
/dev/sdb:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.3.00
Orig Family : 226cc5df
Family : 226cc5df
Generation : 000019dc
Attributes : All supported
UUID : b67ea029:aaea7da2:2540c1eb:ebe98af1
Checksum : 651263bf correct
MPB Sectors : 2
Disks : 4
RAID Devices : 1
Disk02 Serial : Z1E1RPA9
State : active
Id : 00030000
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
[Volume0]:
UUID : 51a415ba:dc9c8cd7:5b3ea8de:465b4630
RAID Level : 5
Members : 4
Slots : [__U_]
Failed disk : 0
This Slot : 2
Array Size : 11721072640 (5589.04 GiB 6001.19 GB)
Per Dev Size : 3907024648 (1863.01 GiB 2000.40 GB)
Sector Offset : 0
Num Stripes : 15261814
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : failed
Dirty State : clean
Disk00 Serial : Z1E1AKPH:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
Disk01 Serial : Z24091Q5:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
Disk03 Serial : Z1E19E4K:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
/dev/sdc:
MBR Magic : aa55
Partition[0] : 3907027057 sectors at 63 (type 42)
/dev/sdd:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.3.00
Orig Family : 226cc5df
Family : 226cc5df
Generation : 000019d9
Attributes : All supported
UUID : b67ea029:aaea7da2:2540c1eb:ebe98af1
Checksum : 641438ba correct
MPB Sectors : 2
Disks : 4
RAID Devices : 1
Disk03 Serial : Z1E19E4K
State : active
Id : 00020000
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
[Volume0]:
UUID : 51a415ba:dc9c8cd7:5b3ea8de:465b4630
RAID Level : 5
Members : 4
Slots : [__UU]
Failed disk : 0
This Slot : 3
Array Size : 11721072640 (5589.04 GiB 6001.19 GB)
Per Dev Size : 3907024648 (1863.01 GiB 2000.40 GB)
Sector Offset : 0
Num Stripes : 15261814
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : failed
Dirty State : clean
Disk00 Serial : Z1E1AKPH:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
Disk01 Serial : Z24091Q5:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
Disk02 Serial : Z1E1RPA9
State : active
Id : 00030000
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
/dev/sde:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
# dpkg -l | grep mdadm
ii mdadm 3.2.5-1+b1
thanks
chris
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-10 16:23 Recovery/Access of imsm raid via mdadm? chris
@ 2013-01-10 17:09 ` Dave Jiang
2013-01-10 20:19 ` chris
0 siblings, 1 reply; 18+ messages in thread
From: Dave Jiang @ 2013-01-10 17:09 UTC (permalink / raw)
To: chris; +Cc: Linux-RAID
On 01/10/2013 09:23 AM, chris wrote:
> Hello,
>
> I have a machine which was running a imsm raid volume, where the
> motherboard failed and I do not have access to another system with
> imsm. I remember noticing some time ago that mdadm could recognize
> these arrays, so I decided to try recovery in a spare machine with the
> disks from the array.
>
> I guess my questions are:
> Is this the right forum for help with this?
> Am I even going down a feasible path here or is this array dependent
> on the HBA in some way?
> If it is possible any ideas of anything else I can do to debug this further?
Typically mdadm probes the OROM and look for platform details. But you
can try overriding with:
export IMSM_NO_PLATFORM=1
See if that works for you.
> The original array was a raid 5 of 4x2TB sata disks
>
> When I examine the first disk, things look good:
>
> mdadm --examine /dev/sdb
> mdadm: /dev/sdb is not attached to Intel(R) RAID controller.
> mdadm: /dev/sdb is not attached to Intel(R) RAID controller.
> /dev/sdb:
> Magic : Intel Raid ISM Cfg Sig.
> Version : 1.3.00
> Orig Family : 226cc5df
> Family : 226cc5df
> Generation : 000019dc
> Attributes : All supported
> UUID : b67ea029:aaea7da2:2540c1eb:ebe98af1
> Checksum : 651263bf correct
> MPB Sectors : 2
> Disks : 4
> RAID Devices : 1
>
> Disk02 Serial : Z1E1RPA9
> State : active
> Id : 00030000
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> [Volume0]:
> UUID : 51a415ba:dc9c8cd7:5b3ea8de:465b4630
> RAID Level : 5
> Members : 4
> Slots : [__U_]
> Failed disk : 0
> This Slot : 2
> Array Size : 11721072640 (5589.04 GiB 6001.19 GB)
> Per Dev Size : 3907024648 (1863.01 GiB 2000.40 GB)
> Sector Offset : 0
> Num Stripes : 15261814
> Chunk Size : 128 KiB
> Reserved : 0
> Migrate State : idle
> Map State : failed
> Dirty State : clean
>
> Disk00 Serial : Z1E1AKPH:0
> State : active
> Id : ffffffff
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> Disk01 Serial : Z24091Q5:0
> State : active
> Id : ffffffff
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> Disk03 Serial : Z1E19E4K:0
> State : active
> Id : ffffffff
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
>
> When I try to scan for arrays I get this:
> # mdadm --examine --scan
> HBAs of devices does not match (null) != (null)
> ARRAY metadata=imsm UUID=b67ea029:aaea7da2:2540c1eb:ebe98af1
> ARRAY /dev/md/Volume0 container=b67ea029:aaea7da2:2540c1eb:ebe98af1
> member=0 UUID=51a415ba:dc9c8cd7:5b3ea8de:465b4630
> ARRAY metadata=imsm UUID=b67ea029:aaea7da2:2540c1eb:ebe98af1
> ARRAY /dev/md/Volume0 container=b67ea029:aaea7da2:2540c1eb:ebe98af1
> member=0 UUID=51a415ba:dc9c8cd7:5b3ea8de:465b4630
>
> My first concern is the warning that the HBA is missing, the whole
> reason I am going at it this way is because I don't have the HBA.
> Second concern is duplicate detection of the same array.
>
> If i try to run # mdadm -As:
> mdadm: No arrays found in config file or automatically
>
> I also tried adding the output from --examine --scan to
> /etc/mdadm/mdadm.conf but after trying that I now get blank output:
>
> # mdadm --assemble /dev/md/Volume0
> #
> # mdadm --assemble --scan
> #
>
> full examine of all disks involved:
>
> /dev/sdb:
> Magic : Intel Raid ISM Cfg Sig.
> Version : 1.3.00
> Orig Family : 226cc5df
> Family : 226cc5df
> Generation : 000019dc
> Attributes : All supported
> UUID : b67ea029:aaea7da2:2540c1eb:ebe98af1
> Checksum : 651263bf correct
> MPB Sectors : 2
> Disks : 4
> RAID Devices : 1
>
> Disk02 Serial : Z1E1RPA9
> State : active
> Id : 00030000
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> [Volume0]:
> UUID : 51a415ba:dc9c8cd7:5b3ea8de:465b4630
> RAID Level : 5
> Members : 4
> Slots : [__U_]
> Failed disk : 0
> This Slot : 2
> Array Size : 11721072640 (5589.04 GiB 6001.19 GB)
> Per Dev Size : 3907024648 (1863.01 GiB 2000.40 GB)
> Sector Offset : 0
> Num Stripes : 15261814
> Chunk Size : 128 KiB
> Reserved : 0
> Migrate State : idle
> Map State : failed
> Dirty State : clean
>
> Disk00 Serial : Z1E1AKPH:0
> State : active
> Id : ffffffff
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> Disk01 Serial : Z24091Q5:0
> State : active
> Id : ffffffff
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> Disk03 Serial : Z1E19E4K:0
> State : active
> Id : ffffffff
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
> /dev/sdc:
> MBR Magic : aa55
> Partition[0] : 3907027057 sectors at 63 (type 42)
> /dev/sdd:
> Magic : Intel Raid ISM Cfg Sig.
> Version : 1.3.00
> Orig Family : 226cc5df
> Family : 226cc5df
> Generation : 000019d9
> Attributes : All supported
> UUID : b67ea029:aaea7da2:2540c1eb:ebe98af1
> Checksum : 641438ba correct
> MPB Sectors : 2
> Disks : 4
> RAID Devices : 1
>
> Disk03 Serial : Z1E19E4K
> State : active
> Id : 00020000
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> [Volume0]:
> UUID : 51a415ba:dc9c8cd7:5b3ea8de:465b4630
> RAID Level : 5
> Members : 4
> Slots : [__UU]
> Failed disk : 0
> This Slot : 3
> Array Size : 11721072640 (5589.04 GiB 6001.19 GB)
> Per Dev Size : 3907024648 (1863.01 GiB 2000.40 GB)
> Sector Offset : 0
> Num Stripes : 15261814
> Chunk Size : 128 KiB
> Reserved : 0
> Migrate State : idle
> Map State : failed
> Dirty State : clean
>
> Disk00 Serial : Z1E1AKPH:0
> State : active
> Id : ffffffff
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> Disk01 Serial : Z24091Q5:0
> State : active
> Id : ffffffff
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
>
> Disk02 Serial : Z1E1RPA9
> State : active
> Id : 00030000
> Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
> /dev/sde:
> MBR Magic : aa55
> Partition[0] : 4294967295 sectors at 1 (type ee)
>
> # dpkg -l | grep mdadm
> ii mdadm 3.2.5-1+b1
>
> thanks
> chris
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-10 17:09 ` Dave Jiang
@ 2013-01-10 20:19 ` chris
2013-01-11 1:42 ` Dan Williams
0 siblings, 1 reply; 18+ messages in thread
From: chris @ 2013-01-10 20:19 UTC (permalink / raw)
To: Dave Jiang; +Cc: Linux-RAID
Hi Dave,
Thanks for the tip, this has gotten me further
# mdadm --assemble --scan
# cat /proc/mdstat
Personalities :
md127 : inactive sdd[1](S) sdb[0](S)
4520 blocks super external:imsm
So atleast now it has created an array of sorts but it only has 2 out
of the 4 disks it should and also all disks appear to be marked as
spares.
Would I follow the same logic as typical mdadm recovery where I need
to create the array again with the right layout and disks in right
order and gain access to data without wiping it ?
Any advice what direction to go in from here?
This info I was able to get using mdadm seems accurate as far as how
the array was setup so I just need to know how to recreate or assemble
the array and get it online so I can access it
Magic : Intel Raid ISM Cfg Sig.
Version : 1.3.00
Orig Family : 226cc5df
Family : 226cc5df
Generation : 000019dc
Attributes : All supported
UUID : b67ea029:aaea7da2:2540c1eb:ebe98af1
Checksum : 651263bf correct
MPB Sectors : 2
Disks : 4
RAID Devices : 1
Disk02 Serial : Z1E1RPA9
State : active
Id : 00030000
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
[Volume0]:
UUID : 51a415ba:dc9c8cd7:5b3ea8de:465b4630
RAID Level : 5
Members : 4
Slots : [__U_]
Failed disk : 0
This Slot : 2
Array Size : 11721072640 (5589.04 GiB 6001.19 GB)
Per Dev Size : 3907024648 (1863.01 GiB 2000.40 GB)
Sector Offset : 0
Num Stripes : 15261814
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : failed
Dirty State : clean
Disk00 Serial : Z1E1AKPH:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
Disk01 Serial : Z24091Q5:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
Disk03 Serial : Z1E19E4K:0
State : active
Id : ffffffff
Usable Size : 3907024648 (1863.01 GiB 2000.40 GB)
thanks
chris
On Thu, Jan 10, 2013 at 12:09 PM, Dave Jiang <dave.jiang@intel.com> wrote:
> export IMSM_NO_PLATFORM=1
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-10 20:19 ` chris
@ 2013-01-11 1:42 ` Dan Williams
2013-01-11 17:53 ` chris
0 siblings, 1 reply; 18+ messages in thread
From: Dan Williams @ 2013-01-11 1:42 UTC (permalink / raw)
To: chris; +Cc: Dave Jiang, Linux-RAID
On Thu, Jan 10, 2013 at 12:19 PM, chris <tknchris@gmail.com> wrote:
> Hi Dave,
>
> Thanks for the tip, this has gotten me further
>
> # mdadm --assemble --scan
> # cat /proc/mdstat
> Personalities :
> md127 : inactive sdd[1](S) sdb[0](S)
> 4520 blocks super external:imsm
>
> So atleast now it has created an array of sorts but it only has 2 out
> of the 4 disks it should and also all disks appear to be marked as
> spares.
>
> Would I follow the same logic as typical mdadm recovery where I need
> to create the array again with the right layout and disks in right
> order and gain access to data without wiping it ?
>
> Any advice what direction to go in from here?
>
Backup the disks you have if possible.
Don't recreate the array with all 4 disks be sure to specify a missing
slot. With mdadm-3.2.5 you do have support for creating imsm arrays
with 'missing' slots [1]. Then you can test some recreations to see
if you can find data.
At a minimum you know that sdb and sdd are in slot 2, 3, so the
question is finding a good candidate for slot 0, or 1. It's a bit
concerning that the superblocks for those have been lost, or those
disks weren't moved over? You can look for serial numbers in
/dev/disks/by-id.
--
Dan
[1]: http://marc.info/?l=linux-raid&m=131432484118038&w=2
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-11 1:42 ` Dan Williams
@ 2013-01-11 17:53 ` chris
2013-01-13 19:00 ` chris
0 siblings, 1 reply; 18+ messages in thread
From: chris @ 2013-01-11 17:53 UTC (permalink / raw)
To: Dan Williams; +Cc: Dave Jiang, Linux-RAID
Ok so I looked through the examine data and it appears the order of
the disks was:
sde 0 - Z1E1AKPH
sdc 1 - Z24091Q5
sdb 2 - Z1E1RPA9
sdd 3 - Z1E19E4K
So I was going to try the first possible combination and assemble with
sde/sdc/sdb/missing, this is what I tried:
# export IMSM_NO_PLATFORM=1
# mdadm --create --verbose /dev/md/imsm /dev/sde /dev/sdc /dev/sdd
missing --raid-devices 4 --metadata=imsm
mdadm: /dev/sde appears to be part of a raid array:
level=raid0 devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: /dev/sdc appears to be part of a raid array:
level=raid0 devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: This level does not support missing devices
The example commands I found for creating imsm array were like this:
mdadm --create --verbose /dev/md/imsm /dev/sde /dev/sdc /dev/sdd
missing --raid-devices 4 --metadata=imsm
mdadm --create --verbose /dev/md/Volume0 /dev/md/imsm --raid-devices 4 --level 5
Should I be defining level=5 when creating /dev/md/imsm also? Or
should I only do it for imsm container and omit it from creating
/dev/md/Volume0?
thanks for everything so far I feel like I'm pretty close, once I can
get the arrays up with missing and dont have to worry about
overwriting anything then I'm sure I can work out the right
combination of disks and hopefully the disks with no imsm metadata
arent so damaged I cant recover
chris
On Thu, Jan 10, 2013 at 8:42 PM, Dan Williams <djbw@fb.com> wrote:
>
>>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-11 17:53 ` chris
@ 2013-01-13 19:00 ` chris
2013-01-13 21:05 ` Dan Williams
0 siblings, 1 reply; 18+ messages in thread
From: chris @ 2013-01-13 19:00 UTC (permalink / raw)
To: Neil Brown; +Cc: Dan Williams, Dave Jiang, Linux-RAID
Neil/Dave,
Is it not possible to create imsm container with missing disk?
If not, Is there any way to recreate the array with all disks but
prevent any kind of sync which may overwrite array data?
Any other ideas where to go from here?
thanks
chris
On Fri, Jan 11, 2013 at 12:53 PM, chris <tknchris@gmail.com> wrote:
> Ok so I looked through the examine data and it appears the order of
> the disks was:
>
> sde 0 - Z1E1AKPH
> sdc 1 - Z24091Q5
> sdb 2 - Z1E1RPA9
> sdd 3 - Z1E19E4K
>
> So I was going to try the first possible combination and assemble with
> sde/sdc/sdb/missing, this is what I tried:
>
> # export IMSM_NO_PLATFORM=1
> # mdadm --create --verbose /dev/md/imsm /dev/sde /dev/sdc /dev/sdd
> missing --raid-devices 4 --metadata=imsm
> mdadm: /dev/sde appears to be part of a raid array:
> level=raid0 devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sde.
> mdadm: /dev/sdc appears to be part of a raid array:
> level=raid0 devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdc.
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdd.
> mdadm: size set to 1953511431K
> Continue creating array? y
> mdadm: This level does not support missing devices
>
> The example commands I found for creating imsm array were like this:
> mdadm --create --verbose /dev/md/imsm /dev/sde /dev/sdc /dev/sdd
> missing --raid-devices 4 --metadata=imsm
> mdadm --create --verbose /dev/md/Volume0 /dev/md/imsm --raid-devices 4 --level 5
>
> Should I be defining level=5 when creating /dev/md/imsm also? Or
> should I only do it for imsm container and omit it from creating
> /dev/md/Volume0?
>
> thanks for everything so far I feel like I'm pretty close, once I can
> get the arrays up with missing and dont have to worry about
> overwriting anything then I'm sure I can work out the right
> combination of disks and hopefully the disks with no imsm metadata
> arent so damaged I cant recover
>
> chris
>
> On Thu, Jan 10, 2013 at 8:42 PM, Dan Williams <djbw@fb.com> wrote:
>>
>>>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-13 19:00 ` chris
@ 2013-01-13 21:05 ` Dan Williams
2013-01-14 0:56 ` chris
0 siblings, 1 reply; 18+ messages in thread
From: Dan Williams @ 2013-01-13 21:05 UTC (permalink / raw)
To: chris, Neil Brown; +Cc: Dave Jiang, Linux-RAID
On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
>Neil/Dave,
>
>Is it not possible to create imsm container with missing disk?
>If not, Is there any way to recreate the array with all disks but
>prevent any kind of sync which may overwrite array data?
The example was in that link I sent:
mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l 5
The first command marks all devices as spares. The second creates the
degraded array.
You probably want at least sdb and sdd in there since they have a copy of
the metadata.
--
Dan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-13 21:05 ` Dan Williams
@ 2013-01-14 0:56 ` chris
2013-01-14 12:36 ` Dorau, Lukasz
` (2 more replies)
0 siblings, 3 replies; 18+ messages in thread
From: chris @ 2013-01-14 0:56 UTC (permalink / raw)
To: Dan Williams; +Cc: Neil Brown, Dave Jiang, Linux-RAID
Hi Dan,
OK so the container comes up just fine now:
# mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
/dev/sde --raid-devices 4 --metadata=imsm
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdb.
mdadm: /dev/sdc appears to be part of a raid array:
level=raid0 devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: /dev/sde appears to be part of a raid array:
level=raid0 devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
root@Microknoppix:/mnt/external# cat /proc/mdstat
Personalities :
md126 : inactive sde[3](S) sdd[2](S) sdc[1](S) sdb[0](S)
4420 blocks super external:imsm
Upon review it all looks good
# mdadm --detail /dev/md/imsm
/dev/md/imsm:
Version : imsm
Raid Level : container
Total Devices : 4
Working Devices : 4
Member Arrays :
Number Major Minor RaidDevice
0 8 16 - /dev/sdb
1 8 32 - /dev/sdc
2 8 48 - /dev/sdd
3 8 64 - /dev/sde
Now when I try all the permutations of disks, they all fail with
"mdadm: failed to activate array."
ATTEMPT #1
# export IMSM_NO_PLATFORM=1
# mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
/dev/sde --raid-devices 4 --metadata=imsm
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdb.
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
# mdadm --create --verbose /dev/md/Volume0 /dev/sde missing /dev/sdb
/dev/sdd --raid-devices 4 --level=5
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: super1.x cannot open /dev/sde: Device or resource busy
mdadm: chunk size defaults to 128K
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sde but will be lost or
meaningless after creating array
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdb but will be lost or
meaningless after creating array
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdd but will be lost or
meaningless after creating array
mdadm: size set to 1953511424K
Continue creating array? y
mdadm: Creating array inside imsm container /dev/md/imsm
mdadm: failed to activate array.
# mdadm --stop /dev/md/imsm
mdadm: stopped /dev/md/imsm
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
unused devices: <none>
ATTEMPT #2
# export IMSM_NO_PLATFORM=1
# mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
/dev/sde --raid-devices 4 --metadata=imsm
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdb.
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
# mdadm --create --verbose /dev/md/Volume0 missing /dev/sdc /dev/sdb
/dev/sdd --raid-devices 4 --level=5
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: super1.x cannot open /dev/sdc: Device or resource busy
mdadm: chunk size defaults to 128K
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdc but will be lost or
meaningless after creating array
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdb but will be lost or
meaningless after creating array
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdd but will be lost or
meaningless after creating array
mdadm: size set to 1953511424K
Continue creating array? y
mdadm: Creating array inside imsm container /dev/md/imsm
mdadm: failed to activate array.
# mdadm --stop /dev/md/imsm
mdadm: stopped /dev/md/imsm
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
unused devices: <none>
ATTEMPT #3
# export IMSM_NO_PLATFORM=1
# mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
/dev/sde --raid-devices 4 --metadata=imsm
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdb.
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
# mdadm --create --verbose /dev/md/Volume0 /dev/sde /dev/sdc missing
/dev/sdd --raid-devices 4 --level=5
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: super1.x cannot open /dev/sde: Device or resource busy
mdadm: chunk size defaults to 128K
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sde but will be lost or
meaningless after creating array
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdc but will be lost or
meaningless after creating array
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdd but will be lost or
meaningless after creating array
mdadm: size set to 1953511424K
Continue creating array? y
mdadm: Creating array inside imsm container /dev/md/imsm
mdadm: failed to activate array.
# mdadm --stop /dev/md/imsm
mdadm: stopped /dev/md/imsm
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
unused devices: <none>
ATTEMPT #4
# export IMSM_NO_PLATFORM=1
# mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
/dev/sde --raid-devices 4 --metadata=imsm
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdb.
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
# mdadm --create --verbose /dev/md/Volume0 /dev/sde /dev/sdc /dev/sdb
missing --raid-devices 4 --level=5
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: super1.x cannot open /dev/sde: Device or resource busy
mdadm: chunk size defaults to 128K
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sde but will be lost or
meaningless after creating array
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdc but will be lost or
meaningless after creating array
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdb but will be lost or
meaningless after creating array
mdadm: size set to 1953511424K
Continue creating array? y
mdadm: Creating array inside imsm container /dev/md/imsm
mdadm: failed to activate array.
Also noticed this in dmesg, all the attemps generate the same output
but heres the dmesg output related to attempt #4:
[292271.903919] md: md127 stopped.
[292271.903938] md: unbind<sde>
[292271.914179] md: export_rdev(sde)
[292271.914424] md: unbind<sdd>
[292271.927499] md: export_rdev(sdd)
[292271.927728] md: unbind<sdc>
[292271.940823] md: export_rdev(sdc)
[292271.940979] md: unbind<sdb>
[292271.954153] md: export_rdev(sdb)
[292289.371545] md: bind<sdb>
[292289.371731] md: bind<sdc>
[292289.371877] md: bind<sdd>
[292289.372024] md: bind<sde>
[292295.910881] md: bind<sde>
[292295.911130] md: bind<sdc>
[292295.911314] md: bind<sdb>
[292295.923942] bio: create slab <bio-1> at 1
[292295.923965] md/raid:md126: not clean -- starting background reconstruction
[292295.924000] md/raid:md126: device sdb operational as raid disk 2
[292295.924005] md/raid:md126: device sdc operational as raid disk 1
[292295.924009] md/raid:md126: device sde operational as raid disk 0
[292295.925149] md/raid:md126: allocated 4250kB
[292295.927268] md/raid:md126: cannot start dirty degraded array.
[292295.929666] RAID conf printout:
[292295.929677] --- level:5 rd:4 wd:3
[292295.929683] disk 0, o:1, dev:sde
[292295.929688] disk 1, o:1, dev:sdc
[292295.929693] disk 2, o:1, dev:sdb
[292295.930898] md/raid:md126: failed to run raid set.
[292295.930902] md: pers->run() failed ...
[292295.931079] md: md126 stopped.
[292295.931096] md: unbind<sdb>
[292295.944228] md: export_rdev(sdb)
[292295.944267] md: unbind<sdc>
[292295.958126] md: export_rdev(sdc)
[292295.958167] md: unbind<sde>
[292295.970902] md: export_rdev(sde)
[292296.219837] device-mapper: table: 252:1: raid45: unknown target type
[292296.219845] device-mapper: ioctl: error adding target to table
[292296.291542] device-mapper: table: 252:1: raid45: unknown target type
[292296.291548] device-mapper: ioctl: error adding target to table
[292296.310926] quiet_error: 1116 callbacks suppressed
[292296.310934] Buffer I/O error on device dm-0, logical block 3907022720
[292296.310940] Buffer I/O error on device dm-0, logical block 3907022721
[292296.310944] Buffer I/O error on device dm-0, logical block 3907022722
[292296.310949] Buffer I/O error on device dm-0, logical block 3907022723
[292296.310953] Buffer I/O error on device dm-0, logical block 3907022724
[292296.310958] Buffer I/O error on device dm-0, logical block 3907022725
[292296.310962] Buffer I/O error on device dm-0, logical block 3907022726
[292296.310966] Buffer I/O error on device dm-0, logical block 3907022727
[292296.310973] Buffer I/O error on device dm-0, logical block 3907022720
[292296.310977] Buffer I/O error on device dm-0, logical block 3907022721
[292296.319968] device-mapper: table: 252:1: raid45: unknown target type
[292296.319975] device-mapper: ioctl: error adding target to table
Any ideas from here? Am I up the creek without a paddle? :(
thanks to everyone for all your help so far
chris
On Sun, Jan 13, 2013 at 4:05 PM, Dan Williams <djbw@fb.com> wrote:
>
>
> On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
>
>>Neil/Dave,
>>
>>Is it not possible to create imsm container with missing disk?
>>If not, Is there any way to recreate the array with all disks but
>>prevent any kind of sync which may overwrite array data?
>
> The example was in that link I sent:
>
> mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
> mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l 5
>
> The first command marks all devices as spares. The second creates the
> degraded array.
>
> You probably want at least sdb and sdd in there since they have a copy of
> the metadata.
>
> --
> Dan
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* RE: Recovery/Access of imsm raid via mdadm?
2013-01-14 0:56 ` chris
@ 2013-01-14 12:36 ` Dorau, Lukasz
2013-01-14 14:10 ` Dorau, Lukasz
2013-01-14 14:24 ` Dorau, Lukasz
2 siblings, 0 replies; 18+ messages in thread
From: Dorau, Lukasz @ 2013-01-14 12:36 UTC (permalink / raw)
To: chris; +Cc: Dan Williams, Neil Brown, Jiang, Dave, Linux-RAID
On Monday, January 14, 2013 1:56 AM chris <tknchris@gmail.com> wrote:
>
> Now when I try all the permutations of disks, they all fail with
> "mdadm: failed to activate array."
>
There are 2 more possibilities you should try:
# mdadm --create --verbose /dev/md/Volume0 /dev/sdc missing /dev/sdb /dev/sdd --raid-devices 4 --level=5
and
# mdadm --create --verbose /dev/md/Volume0 missing /dev/sde /dev/sdb /dev/sdd --raid-devices 4 --level=5
Lukasz
> ATTEMPT #1
> # export IMSM_NO_PLATFORM=1
> # mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
> /dev/sde --raid-devices 4 --metadata=imsm
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdb.
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdc.
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdd.
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sde.
> mdadm: size set to 1953511431K
> Continue creating array? y
> mdadm: container /dev/md/imsm prepared.
> # mdadm --create --verbose /dev/md/Volume0 /dev/sde missing /dev/sdb
> /dev/sdd --raid-devices 4 --level=5
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: super1.x cannot open /dev/sde: Device or resource busy
> mdadm: chunk size defaults to 128K
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sde but will be lost or
> meaningless after creating array
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdb but will be lost or
> meaningless after creating array
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdd but will be lost or
> meaningless after creating array
> mdadm: size set to 1953511424K
> Continue creating array? y
> mdadm: Creating array inside imsm container /dev/md/imsm
> mdadm: failed to activate array.
> # mdadm --stop /dev/md/imsm
> mdadm: stopped /dev/md/imsm
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> unused devices: <none>
>
> ATTEMPT #2
> # export IMSM_NO_PLATFORM=1
> # mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
> /dev/sde --raid-devices 4 --metadata=imsm
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdb.
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdc.
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdd.
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sde.
> mdadm: size set to 1953511431K
> Continue creating array? y
> mdadm: container /dev/md/imsm prepared.
> # mdadm --create --verbose /dev/md/Volume0 missing /dev/sdc /dev/sdb
> /dev/sdd --raid-devices 4 --level=5
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: super1.x cannot open /dev/sdc: Device or resource busy
> mdadm: chunk size defaults to 128K
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdc but will be lost or
> meaningless after creating array
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdb but will be lost or
> meaningless after creating array
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdd but will be lost or
> meaningless after creating array
> mdadm: size set to 1953511424K
> Continue creating array? y
> mdadm: Creating array inside imsm container /dev/md/imsm
> mdadm: failed to activate array.
> # mdadm --stop /dev/md/imsm
> mdadm: stopped /dev/md/imsm
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> unused devices: <none>
>
> ATTEMPT #3
> # export IMSM_NO_PLATFORM=1
> # mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
> /dev/sde --raid-devices 4 --metadata=imsm
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdb.
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdc.
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdd.
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sde.
> mdadm: size set to 1953511431K
> Continue creating array? y
> mdadm: container /dev/md/imsm prepared.
> # mdadm --create --verbose /dev/md/Volume0 /dev/sde /dev/sdc missing
> /dev/sdd --raid-devices 4 --level=5
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: super1.x cannot open /dev/sde: Device or resource busy
> mdadm: chunk size defaults to 128K
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sde but will be lost or
> meaningless after creating array
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdc but will be lost or
> meaningless after creating array
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdd but will be lost or
> meaningless after creating array
> mdadm: size set to 1953511424K
> Continue creating array? y
> mdadm: Creating array inside imsm container /dev/md/imsm
> mdadm: failed to activate array.
> # mdadm --stop /dev/md/imsm
> mdadm: stopped /dev/md/imsm
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> unused devices: <none>
>
> ATTEMPT #4
> # export IMSM_NO_PLATFORM=1
> # mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
> /dev/sde --raid-devices 4 --metadata=imsm
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdb.
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdc.
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdd.
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sde.
> mdadm: size set to 1953511431K
> Continue creating array? y
> mdadm: container /dev/md/imsm prepared.
> # mdadm --create --verbose /dev/md/Volume0 /dev/sde /dev/sdc /dev/sdb
> missing --raid-devices 4 --level=5
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: super1.x cannot open /dev/sde: Device or resource busy
> mdadm: chunk size defaults to 128K
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sde but will be lost or
> meaningless after creating array
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdc but will be lost or
> meaningless after creating array
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdb but will be lost or
> meaningless after creating array
> mdadm: size set to 1953511424K
> Continue creating array? y
> mdadm: Creating array inside imsm container /dev/md/imsm
> mdadm: failed to activate array.
>
> Also noticed this in dmesg, all the attemps generate the same output
> but heres the dmesg output related to attempt #4:
>
> [292271.903919] md: md127 stopped.
> [292271.903938] md: unbind<sde>
> [292271.914179] md: export_rdev(sde)
> [292271.914424] md: unbind<sdd>
> [292271.927499] md: export_rdev(sdd)
> [292271.927728] md: unbind<sdc>
> [292271.940823] md: export_rdev(sdc)
> [292271.940979] md: unbind<sdb>
> [292271.954153] md: export_rdev(sdb)
> [292289.371545] md: bind<sdb>
> [292289.371731] md: bind<sdc>
> [292289.371877] md: bind<sdd>
> [292289.372024] md: bind<sde>
> [292295.910881] md: bind<sde>
> [292295.911130] md: bind<sdc>
> [292295.911314] md: bind<sdb>
> [292295.923942] bio: create slab <bio-1> at 1
> [292295.923965] md/raid:md126: not clean -- starting background
> reconstruction
> [292295.924000] md/raid:md126: device sdb operational as raid disk 2
> [292295.924005] md/raid:md126: device sdc operational as raid disk 1
> [292295.924009] md/raid:md126: device sde operational as raid disk 0
> [292295.925149] md/raid:md126: allocated 4250kB
> [292295.927268] md/raid:md126: cannot start dirty degraded array.
> [292295.929666] RAID conf printout:
> [292295.929677] --- level:5 rd:4 wd:3
> [292295.929683] disk 0, o:1, dev:sde
> [292295.929688] disk 1, o:1, dev:sdc
> [292295.929693] disk 2, o:1, dev:sdb
> [292295.930898] md/raid:md126: failed to run raid set.
> [292295.930902] md: pers->run() failed ...
> [292295.931079] md: md126 stopped.
> [292295.931096] md: unbind<sdb>
> [292295.944228] md: export_rdev(sdb)
> [292295.944267] md: unbind<sdc>
> [292295.958126] md: export_rdev(sdc)
> [292295.958167] md: unbind<sde>
> [292295.970902] md: export_rdev(sde)
> [292296.219837] device-mapper: table: 252:1: raid45: unknown target type
> [292296.219845] device-mapper: ioctl: error adding target to table
> [292296.291542] device-mapper: table: 252:1: raid45: unknown target type
> [292296.291548] device-mapper: ioctl: error adding target to table
> [292296.310926] quiet_error: 1116 callbacks suppressed
> [292296.310934] Buffer I/O error on device dm-0, logical block 3907022720
> [292296.310940] Buffer I/O error on device dm-0, logical block 3907022721
> [292296.310944] Buffer I/O error on device dm-0, logical block 3907022722
> [292296.310949] Buffer I/O error on device dm-0, logical block 3907022723
> [292296.310953] Buffer I/O error on device dm-0, logical block 3907022724
> [292296.310958] Buffer I/O error on device dm-0, logical block 3907022725
> [292296.310962] Buffer I/O error on device dm-0, logical block 3907022726
> [292296.310966] Buffer I/O error on device dm-0, logical block 3907022727
> [292296.310973] Buffer I/O error on device dm-0, logical block 3907022720
> [292296.310977] Buffer I/O error on device dm-0, logical block 3907022721
> [292296.319968] device-mapper: table: 252:1: raid45: unknown target type
> [292296.319975] device-mapper: ioctl: error adding target to table
>
> Any ideas from here? Am I up the creek without a paddle? :(
>
> thanks to everyone for all your help so far
> chris
>
> On Sun, Jan 13, 2013 at 4:05 PM, Dan Williams <djbw@fb.com> wrote:
> >
> >
> > On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
> >
> >>Neil/Dave,
> >>
> >>Is it not possible to create imsm container with missing disk?
> >>If not, Is there any way to recreate the array with all disks but
> >>prevent any kind of sync which may overwrite array data?
> >
> > The example was in that link I sent:
> >
> > mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
> > mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l 5
> >
> > The first command marks all devices as spares. The second creates the
> > degraded array.
> >
> > You probably want at least sdb and sdd in there since they have a copy of
> > the metadata.
> >
> > --
> > Dan
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* RE: Recovery/Access of imsm raid via mdadm?
2013-01-14 0:56 ` chris
2013-01-14 12:36 ` Dorau, Lukasz
@ 2013-01-14 14:10 ` Dorau, Lukasz
2013-01-14 14:24 ` Dorau, Lukasz
2 siblings, 0 replies; 18+ messages in thread
From: Dorau, Lukasz @ 2013-01-14 14:10 UTC (permalink / raw)
To: chris; +Cc: Neil Brown, Jiang, Dave, Linux-RAID, Dan Williams
On Monday, January 14, 2013 1:56 AM chris <tknchris@gmail.com> wrote:
> [292295.923942] bio: create slab <bio-1> at 1
> [292295.923965] md/raid:md126: not clean -- starting background
> reconstruction
> [292295.924000] md/raid:md126: device sdb operational as raid disk 2
> [292295.924005] md/raid:md126: device sdc operational as raid disk 1
> [292295.924009] md/raid:md126: device sde operational as raid disk 0
> [292295.925149] md/raid:md126: allocated 4250kB
> [292295.927268] md/raid:md126: cannot start dirty degraded array.
Hi
*Remember to backup the disks you have before trying the following! *
You can try starting dirty degraded array using:
# mdadm --assemble --force ....
See also the "Boot time assembly of degraded/dirty arrays" chapter in:
http://www.kernel.org/doc/Documentation/md.txt
(you can boot with option md-mod.start_dirty_degraded=1)
Lukasz
> [292295.929666] RAID conf printout:
> [292295.929677] --- level:5 rd:4 wd:3
> [292295.929683] disk 0, o:1, dev:sde
> [292295.929688] disk 1, o:1, dev:sdc
> [292295.929693] disk 2, o:1, dev:sdb
> [292295.930898] md/raid:md126: failed to run raid set.
> [292295.930902] md: pers->run() failed ...
> [292295.931079] md: md126 stopped.
> [292295.931096] md: unbind<sdb>
> [292295.944228] md: export_rdev(sdb)
> [292295.944267] md: unbind<sdc>
> [292295.958126] md: export_rdev(sdc)
> [292295.958167] md: unbind<sde>
> [292295.970902] md: export_rdev(sde)
> [292296.219837] device-mapper: table: 252:1: raid45: unknown target type
> [292296.219845] device-mapper: ioctl: error adding target to table
> [292296.291542] device-mapper: table: 252:1: raid45: unknown target type
> [292296.291548] device-mapper: ioctl: error adding target to table
> [292296.310926] quiet_error: 1116 callbacks suppressed
> [292296.310934] Buffer I/O error on device dm-0, logical block 3907022720
> [292296.310940] Buffer I/O error on device dm-0, logical block 3907022721
> [292296.310944] Buffer I/O error on device dm-0, logical block 3907022722
> [292296.310949] Buffer I/O error on device dm-0, logical block 3907022723
> [292296.310953] Buffer I/O error on device dm-0, logical block 3907022724
> [292296.310958] Buffer I/O error on device dm-0, logical block 3907022725
> [292296.310962] Buffer I/O error on device dm-0, logical block 3907022726
> [292296.310966] Buffer I/O error on device dm-0, logical block 3907022727
> [292296.310973] Buffer I/O error on device dm-0, logical block 3907022720
> [292296.310977] Buffer I/O error on device dm-0, logical block 3907022721
> [292296.319968] device-mapper: table: 252:1: raid45: unknown target type
> [292296.319975] device-mapper: ioctl: error adding target to table
>
> Any ideas from here? Am I up the creek without a paddle? :(
>
> thanks to everyone for all your help so far
> chris
>
> On Sun, Jan 13, 2013 at 4:05 PM, Dan Williams <djbw@fb.com> wrote:
> >
> >
> > On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
> >
> >>Neil/Dave,
> >>
> >>Is it not possible to create imsm container with missing disk?
> >>If not, Is there any way to recreate the array with all disks but
> >>prevent any kind of sync which may overwrite array data?
> >
> > The example was in that link I sent:
> >
> > mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
> > mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l 5
> >
> > The first command marks all devices as spares. The second creates the
> > degraded array.
> >
> > You probably want at least sdb and sdd in there since they have a copy of
> > the metadata.
> >
> > --
> > Dan
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* RE: Recovery/Access of imsm raid via mdadm?
2013-01-14 0:56 ` chris
2013-01-14 12:36 ` Dorau, Lukasz
2013-01-14 14:10 ` Dorau, Lukasz
@ 2013-01-14 14:24 ` Dorau, Lukasz
2013-01-14 15:25 ` chris
2 siblings, 1 reply; 18+ messages in thread
From: Dorau, Lukasz @ 2013-01-14 14:24 UTC (permalink / raw)
To: chris; +Cc: Neil Brown, Jiang, Dave, Linux-RAID, Dan Williams
On Monday, January 14, 2013 3:11 PM Dorau, Lukasz <lukasz.dorau@intel.com> wrote:
> On Monday, January 14, 2013 1:56 AM chris <tknchris@gmail.com> wrote:
> > [292295.923942] bio: create slab <bio-1> at 1
> > [292295.923965] md/raid:md126: not clean -- starting background
> > reconstruction
> > [292295.924000] md/raid:md126: device sdb operational as raid disk 2
> > [292295.924005] md/raid:md126: device sdc operational as raid disk 1
> > [292295.924009] md/raid:md126: device sde operational as raid disk 0
> > [292295.925149] md/raid:md126: allocated 4250kB
> > [292295.927268] md/raid:md126: cannot start dirty degraded array.
>
> Hi
>
> *Remember to backup the disks you have before trying the following! *
>
> You can try starting dirty degraded array using:
> # mdadm --assemble --force ....
>
I meant adding --force option to:
# mdadm --create --verbose --force /dev/md/Volume0 /dev/sdc missing /dev/sdb /dev/sdd --raid-devices 4 --level=5
Be very careful using "--force" option, because it can cause data corruption!
Lukasz
> See also the "Boot time assembly of degraded/dirty arrays" chapter in:
> http://www.kernel.org/doc/Documentation/md.txt
> (you can boot with option md-mod.start_dirty_degraded=1)
>
> Lukasz
>
>
> > [292295.929666] RAID conf printout:
> > [292295.929677] --- level:5 rd:4 wd:3
> > [292295.929683] disk 0, o:1, dev:sde
> > [292295.929688] disk 1, o:1, dev:sdc
> > [292295.929693] disk 2, o:1, dev:sdb
> > [292295.930898] md/raid:md126: failed to run raid set.
> > [292295.930902] md: pers->run() failed ...
> > [292295.931079] md: md126 stopped.
> > [292295.931096] md: unbind<sdb>
> > [292295.944228] md: export_rdev(sdb)
> > [292295.944267] md: unbind<sdc>
> > [292295.958126] md: export_rdev(sdc)
> > [292295.958167] md: unbind<sde>
> > [292295.970902] md: export_rdev(sde)
> > [292296.219837] device-mapper: table: 252:1: raid45: unknown target type
> > [292296.219845] device-mapper: ioctl: error adding target to table
> > [292296.291542] device-mapper: table: 252:1: raid45: unknown target type
> > [292296.291548] device-mapper: ioctl: error adding target to table
> > [292296.310926] quiet_error: 1116 callbacks suppressed
> > [292296.310934] Buffer I/O error on device dm-0, logical block 3907022720
> > [292296.310940] Buffer I/O error on device dm-0, logical block 3907022721
> > [292296.310944] Buffer I/O error on device dm-0, logical block 3907022722
> > [292296.310949] Buffer I/O error on device dm-0, logical block 3907022723
> > [292296.310953] Buffer I/O error on device dm-0, logical block 3907022724
> > [292296.310958] Buffer I/O error on device dm-0, logical block 3907022725
> > [292296.310962] Buffer I/O error on device dm-0, logical block 3907022726
> > [292296.310966] Buffer I/O error on device dm-0, logical block 3907022727
> > [292296.310973] Buffer I/O error on device dm-0, logical block 3907022720
> > [292296.310977] Buffer I/O error on device dm-0, logical block 3907022721
> > [292296.319968] device-mapper: table: 252:1: raid45: unknown target type
> > [292296.319975] device-mapper: ioctl: error adding target to table
> >
> > Any ideas from here? Am I up the creek without a paddle? :(
> >
> > thanks to everyone for all your help so far
> > chris
> >
> > On Sun, Jan 13, 2013 at 4:05 PM, Dan Williams <djbw@fb.com> wrote:
> > >
> > >
> > > On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
> > >
> > >>Neil/Dave,
> > >>
> > >>Is it not possible to create imsm container with missing disk?
> > >>If not, Is there any way to recreate the array with all disks but
> > >>prevent any kind of sync which may overwrite array data?
> > >
> > > The example was in that link I sent:
> > >
> > > mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
> > > mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l 5
> > >
> > > The first command marks all devices as spares. The second creates the
> > > degraded array.
> > >
> > > You probably want at least sdb and sdd in there since they have a copy of
> > > the metadata.
> > >
> > > --
> > > Dan
> > >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-14 14:24 ` Dorau, Lukasz
@ 2013-01-14 15:25 ` chris
2013-01-15 10:25 ` Dorau, Lukasz
0 siblings, 1 reply; 18+ messages in thread
From: chris @ 2013-01-14 15:25 UTC (permalink / raw)
To: Dorau, Lukasz; +Cc: Neil Brown, Jiang, Dave, Linux-RAID, Dan Williams
Ok thanks for the tips, I am imaging the disks now and will try after
that is done. Just out of curiousity what could become corrupted by
forcing the assemble? I was under the impression that as long as I
have one member missing that the only thing that would be touched is
metadata, is that right?
thanks
chris
On Mon, Jan 14, 2013 at 9:24 AM, Dorau, Lukasz <lukasz.dorau@intel.com> wrote:
> On Monday, January 14, 2013 3:11 PM Dorau, Lukasz <lukasz.dorau@intel.com> wrote:
>> On Monday, January 14, 2013 1:56 AM chris <tknchris@gmail.com> wrote:
>> > [292295.923942] bio: create slab <bio-1> at 1
>> > [292295.923965] md/raid:md126: not clean -- starting background
>> > reconstruction
>> > [292295.924000] md/raid:md126: device sdb operational as raid disk 2
>> > [292295.924005] md/raid:md126: device sdc operational as raid disk 1
>> > [292295.924009] md/raid:md126: device sde operational as raid disk 0
>> > [292295.925149] md/raid:md126: allocated 4250kB
>> > [292295.927268] md/raid:md126: cannot start dirty degraded array.
>>
>> Hi
>>
>> *Remember to backup the disks you have before trying the following! *
>>
>> You can try starting dirty degraded array using:
>> # mdadm --assemble --force ....
>>
>
> I meant adding --force option to:
> # mdadm --create --verbose --force /dev/md/Volume0 /dev/sdc missing /dev/sdb /dev/sdd --raid-devices 4 --level=5
>
> Be very careful using "--force" option, because it can cause data corruption!
>
> Lukasz
>
>
>> See also the "Boot time assembly of degraded/dirty arrays" chapter in:
>> http://www.kernel.org/doc/Documentation/md.txt
>> (you can boot with option md-mod.start_dirty_degraded=1)
>>
>> Lukasz
>>
>>
>> > [292295.929666] RAID conf printout:
>> > [292295.929677] --- level:5 rd:4 wd:3
>> > [292295.929683] disk 0, o:1, dev:sde
>> > [292295.929688] disk 1, o:1, dev:sdc
>> > [292295.929693] disk 2, o:1, dev:sdb
>> > [292295.930898] md/raid:md126: failed to run raid set.
>> > [292295.930902] md: pers->run() failed ...
>> > [292295.931079] md: md126 stopped.
>> > [292295.931096] md: unbind<sdb>
>> > [292295.944228] md: export_rdev(sdb)
>> > [292295.944267] md: unbind<sdc>
>> > [292295.958126] md: export_rdev(sdc)
>> > [292295.958167] md: unbind<sde>
>> > [292295.970902] md: export_rdev(sde)
>> > [292296.219837] device-mapper: table: 252:1: raid45: unknown target type
>> > [292296.219845] device-mapper: ioctl: error adding target to table
>> > [292296.291542] device-mapper: table: 252:1: raid45: unknown target type
>> > [292296.291548] device-mapper: ioctl: error adding target to table
>> > [292296.310926] quiet_error: 1116 callbacks suppressed
>> > [292296.310934] Buffer I/O error on device dm-0, logical block 3907022720
>> > [292296.310940] Buffer I/O error on device dm-0, logical block 3907022721
>> > [292296.310944] Buffer I/O error on device dm-0, logical block 3907022722
>> > [292296.310949] Buffer I/O error on device dm-0, logical block 3907022723
>> > [292296.310953] Buffer I/O error on device dm-0, logical block 3907022724
>> > [292296.310958] Buffer I/O error on device dm-0, logical block 3907022725
>> > [292296.310962] Buffer I/O error on device dm-0, logical block 3907022726
>> > [292296.310966] Buffer I/O error on device dm-0, logical block 3907022727
>> > [292296.310973] Buffer I/O error on device dm-0, logical block 3907022720
>> > [292296.310977] Buffer I/O error on device dm-0, logical block 3907022721
>> > [292296.319968] device-mapper: table: 252:1: raid45: unknown target type
>> > [292296.319975] device-mapper: ioctl: error adding target to table
>> >
>> > Any ideas from here? Am I up the creek without a paddle? :(
>> >
>> > thanks to everyone for all your help so far
>> > chris
>> >
>> > On Sun, Jan 13, 2013 at 4:05 PM, Dan Williams <djbw@fb.com> wrote:
>> > >
>> > >
>> > > On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
>> > >
>> > >>Neil/Dave,
>> > >>
>> > >>Is it not possible to create imsm container with missing disk?
>> > >>If not, Is there any way to recreate the array with all disks but
>> > >>prevent any kind of sync which may overwrite array data?
>> > >
>> > > The example was in that link I sent:
>> > >
>> > > mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
>> > > mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l 5
>> > >
>> > > The first command marks all devices as spares. The second creates the
>> > > degraded array.
>> > >
>> > > You probably want at least sdb and sdd in there since they have a copy of
>> > > the metadata.
>> > >
>> > > --
>> > > Dan
>> > >
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> > the body of a message to majordomo@vger.kernel.org
>> > More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* RE: Recovery/Access of imsm raid via mdadm?
2013-01-14 15:25 ` chris
@ 2013-01-15 10:25 ` Dorau, Lukasz
2013-01-16 16:49 ` chris
0 siblings, 1 reply; 18+ messages in thread
From: Dorau, Lukasz @ 2013-01-15 10:25 UTC (permalink / raw)
To: chris; +Cc: Neil Brown, Jiang, Dave, Linux-RAID, Dan Williams
On Monday, January 14, 2013 4:25 PM chris <tknchris@gmail.com> wrote:
> Ok thanks for the tips, I am imaging the disks now and will try after
> that is done. Just out of curiousity what could become corrupted by
> forcing the assemble? I was under the impression that as long as I
> have one member missing that the only thing that would be touched is
> metadata, is that right?
>
Yes, that is right. I meant that using --force option it may be possible to assembly an array in the wrong way and data can be incorrect, so it is better to be careful.
Lukasz
> On Mon, Jan 14, 2013 at 9:24 AM, Dorau, Lukasz <lukasz.dorau@intel.com>
> wrote:
> > On Monday, January 14, 2013 3:11 PM Dorau, Lukasz
> <lukasz.dorau@intel.com> wrote:
> >> On Monday, January 14, 2013 1:56 AM chris <tknchris@gmail.com> wrote:
> >> > [292295.923942] bio: create slab <bio-1> at 1
> >> > [292295.923965] md/raid:md126: not clean -- starting background
> >> > reconstruction
> >> > [292295.924000] md/raid:md126: device sdb operational as raid disk 2
> >> > [292295.924005] md/raid:md126: device sdc operational as raid disk 1
> >> > [292295.924009] md/raid:md126: device sde operational as raid disk 0
> >> > [292295.925149] md/raid:md126: allocated 4250kB
> >> > [292295.927268] md/raid:md126: cannot start dirty degraded array.
> >>
> >> Hi
> >>
> >> *Remember to backup the disks you have before trying the following! *
> >>
> >> You can try starting dirty degraded array using:
> >> # mdadm --assemble --force ....
> >>
> >
> > I meant adding --force option to:
> > # mdadm --create --verbose --force /dev/md/Volume0 /dev/sdc missing
> /dev/sdb /dev/sdd --raid-devices 4 --level=5
> >
> > Be very careful using "--force" option, because it can cause data corruption!
> >
> > Lukasz
> >
> >
> >> See also the "Boot time assembly of degraded/dirty arrays" chapter in:
> >> http://www.kernel.org/doc/Documentation/md.txt
> >> (you can boot with option md-mod.start_dirty_degraded=1)
> >>
> >> Lukasz
> >>
> >>
> >> > [292295.929666] RAID conf printout:
> >> > [292295.929677] --- level:5 rd:4 wd:3
> >> > [292295.929683] disk 0, o:1, dev:sde
> >> > [292295.929688] disk 1, o:1, dev:sdc
> >> > [292295.929693] disk 2, o:1, dev:sdb
> >> > [292295.930898] md/raid:md126: failed to run raid set.
> >> > [292295.930902] md: pers->run() failed ...
> >> > [292295.931079] md: md126 stopped.
> >> > [292295.931096] md: unbind<sdb>
> >> > [292295.944228] md: export_rdev(sdb)
> >> > [292295.944267] md: unbind<sdc>
> >> > [292295.958126] md: export_rdev(sdc)
> >> > [292295.958167] md: unbind<sde>
> >> > [292295.970902] md: export_rdev(sde)
> >> > [292296.219837] device-mapper: table: 252:1: raid45: unknown target type
> >> > [292296.219845] device-mapper: ioctl: error adding target to table
> >> > [292296.291542] device-mapper: table: 252:1: raid45: unknown target type
> >> > [292296.291548] device-mapper: ioctl: error adding target to table
> >> > [292296.310926] quiet_error: 1116 callbacks suppressed
> >> > [292296.310934] Buffer I/O error on device dm-0, logical block
> 3907022720
> >> > [292296.310940] Buffer I/O error on device dm-0, logical block
> 3907022721
> >> > [292296.310944] Buffer I/O error on device dm-0, logical block
> 3907022722
> >> > [292296.310949] Buffer I/O error on device dm-0, logical block
> 3907022723
> >> > [292296.310953] Buffer I/O error on device dm-0, logical block
> 3907022724
> >> > [292296.310958] Buffer I/O error on device dm-0, logical block
> 3907022725
> >> > [292296.310962] Buffer I/O error on device dm-0, logical block
> 3907022726
> >> > [292296.310966] Buffer I/O error on device dm-0, logical block
> 3907022727
> >> > [292296.310973] Buffer I/O error on device dm-0, logical block
> 3907022720
> >> > [292296.310977] Buffer I/O error on device dm-0, logical block
> 3907022721
> >> > [292296.319968] device-mapper: table: 252:1: raid45: unknown target type
> >> > [292296.319975] device-mapper: ioctl: error adding target to table
> >> >
> >> > Any ideas from here? Am I up the creek without a paddle? :(
> >> >
> >> > thanks to everyone for all your help so far
> >> > chris
> >> >
> >> > On Sun, Jan 13, 2013 at 4:05 PM, Dan Williams <djbw@fb.com> wrote:
> >> > >
> >> > >
> >> > > On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
> >> > >
> >> > >>Neil/Dave,
> >> > >>
> >> > >>Is it not possible to create imsm container with missing disk?
> >> > >>If not, Is there any way to recreate the array with all disks but
> >> > >>prevent any kind of sync which may overwrite array data?
> >> > >
> >> > > The example was in that link I sent:
> >> > >
> >> > > mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
> >> > > mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l
> 5
> >> > >
> >> > > The first command marks all devices as spares. The second creates the
> >> > > degraded array.
> >> > >
> >> > > You probably want at least sdb and sdd in there since they have a copy of
> >> > > the metadata.
> >> > >
> >> > > --
> >> > > Dan
> >> > >
> >> > --
> >> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >> > the body of a message to majordomo@vger.kernel.org
> >> > More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-15 10:25 ` Dorau, Lukasz
@ 2013-01-16 16:49 ` chris
2013-01-16 16:53 ` chris
2013-01-16 22:47 ` Dan Williams
0 siblings, 2 replies; 18+ messages in thread
From: chris @ 2013-01-16 16:49 UTC (permalink / raw)
To: Dorau, Lukasz; +Cc: Neil Brown, Jiang, Dave, Linux-RAID, Dan Williams
Hi, so after painfully imaging 4x2tb I have tried the suggestions of
the other 2 permutations and adding --force with no change
Other 2 permutations:
# mdadm --stop /dev/md/imsm
# export IMSM_NO_PLATFORM=1
# mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
/dev/sde --raid-devices 4 --metadata=imsm
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdb.
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
# mdadm --create --verbose /dev/md/Volume0 /dev/sdc missing /dev/sdb
/dev/sdd --raid-devices 4 --level=5
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: super1.x cannot open /dev/sdc: Device or resource busy
mdadm: chunk size defaults to 128K
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdc but will be lost or
meaningless after creating array
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdb but will be lost or
meaningless after creating array
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdd but will be lost or
meaningless after creating array
mdadm: size set to 1953511424K
Continue creating array? y
mdadm: Creating array inside imsm container /dev/md/imsm
mdadm: failed to activate array.
# mdadm --stop /dev/md/imsm
# export IMSM_NO_PLATFORM=1
# mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
/dev/sde --raid-devices 4 --metadata=imsm
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdb.
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
# mdadm --create --verbose /dev/md/Volume0 missing /dev/sde /dev/sdb
/dev/sdd --raid-devices 4 --level=5
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: super1.x cannot open /dev/sde: Device or resource busy
mdadm: chunk size defaults to 128K
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sde but will be lost or
meaningless after creating array
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdb but will be lost or
meaningless after creating array
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdd but will be lost or
meaningless after creating array
mdadm: size set to 1953511424K
Continue creating array? y
mdadm: Creating array inside imsm container /dev/md/imsm
mdadm: failed to activate array.
Tried again with --force but same thing:
# mdadm --stop /dev/md/imsm
mdadm: stopped /dev/md/imsm
# export IMSM_NO_PLATFORM=1
# mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
/dev/sde --raid-devices 4 --metadata=imsm
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdb.
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdc.
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sdd.
mdadm: /dev/sde appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: metadata will over-write last partition on /dev/sde.
mdadm: size set to 1953511431K
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
# mdadm --create --verbose --force /dev/md/Volume0 /dev/sdc missing
/dev/sdb /dev/sdd --raid-devices 4 --level=5
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: super1.x cannot open /dev/sdc: Device or resource busy
mdadm: chunk size defaults to 128K
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdc but will be lost or
meaningless after creating array
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdb but will be lost or
meaningless after creating array
mdadm: /dev/sdd appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
mdadm: partition table exists on /dev/sdd but will be lost or
meaningless after creating array
mdadm: size set to 1953511424K
Continue creating array? y
mdadm: Creating array inside imsm container /dev/md/imsm
mdadm: failed to activate array.
dmesg shows:
[522342.240901] md: md127 stopped.
[522342.240925] md: unbind<sde>
[522342.251221] md: export_rdev(sde)
[522342.251457] md: unbind<sdd>
[522342.267808] md: export_rdev(sdd)
[522342.267999] md: unbind<sdc>
[522342.281160] md: export_rdev(sdc)
[522342.281309] md: unbind<sdb>
[522342.291136] md: export_rdev(sdb)
[522351.758217] md: bind<sdb>
[522351.758409] md: bind<sdc>
[522351.758552] md: bind<sdd>
[522351.758690] md: bind<sde>
[522368.121090] md: bind<sdc>
[522368.122401] md: bind<sdb>
[522368.122577] md: bind<sdd>
[522368.147454] bio: create slab <bio-1> at 1
[522368.147477] md/raid:md126: not clean -- starting background reconstruction
[522368.147515] md/raid:md126: device sdd operational as raid disk 3
[522368.147520] md/raid:md126: device sdb operational as raid disk 2
[522368.147525] md/raid:md126: device sdc operational as raid disk 0
[522368.148651] md/raid:md126: allocated 4250kB
[522368.152966] md/raid:md126: cannot start dirty degraded array.
[522368.155245] RAID conf printout:
[522368.155259] --- level:5 rd:4 wd:3
[522368.155269] disk 0, o:1, dev:sdc
[522368.155275] disk 2, o:1, dev:sdb
[522368.155281] disk 3, o:1, dev:sdd
[522368.157095] md/raid:md126: failed to run raid set.
[522368.157102] md: pers->run() failed ...
[522368.157418] md: md126 stopped.
[522368.157435] md: unbind<sdd>
[522368.167883] md: export_rdev(sdd)
[522368.167922] md: unbind<sdb>
[522368.181259] md: export_rdev(sdb)
[522368.181302] md: unbind<sdc>
[522368.194576] md: export_rdev(sdc)
[522368.701814] device-mapper: table: 252:1: raid45: unknown target type
[522368.701820] device-mapper: ioctl: error adding target to table
[522368.775341] device-mapper: table: 252:1: raid45: unknown target type
[522368.775347] device-mapper: ioctl: error adding target to table
[522368.876314] quiet_error: 1116 callbacks suppressed
[522368.876324] Buffer I/O error on device dm-0, logical block 3907022720
[522368.876331] Buffer I/O error on device dm-0, logical block 3907022721
[522368.876335] Buffer I/O error on device dm-0, logical block 3907022722
[522368.876340] Buffer I/O error on device dm-0, logical block 3907022723
[522368.876344] Buffer I/O error on device dm-0, logical block 3907022724
[522368.876348] Buffer I/O error on device dm-0, logical block 3907022725
[522368.876352] Buffer I/O error on device dm-0, logical block 3907022726
[522368.876356] Buffer I/O error on device dm-0, logical block 3907022727
[522368.876362] Buffer I/O error on device dm-0, logical block 3907022720
[522368.876366] Buffer I/O error on device dm-0, logical block 3907022721
[522368.883428] device-mapper: table: 252:1: raid45: unknown target type
[522368.883434] device-mapper: ioctl: error adding target to table
[522371.066343] device-mapper: table: 252:1: raid45: unknown target type
[522371.066350] device-mapper: ioctl: error adding target to table
any idea why it wont assemble? I thought even if data was corrupt I
should be able to force it to assemble and look at it to determine if
it is corrupt or in tact
thanks
chris
On Tue, Jan 15, 2013 at 5:25 AM, Dorau, Lukasz <lukasz.dorau@intel.com> wrote:
> On Monday, January 14, 2013 4:25 PM chris <tknchris@gmail.com> wrote:
>> Ok thanks for the tips, I am imaging the disks now and will try after
>> that is done. Just out of curiousity what could become corrupted by
>> forcing the assemble? I was under the impression that as long as I
>> have one member missing that the only thing that would be touched is
>> metadata, is that right?
>>
> Yes, that is right. I meant that using --force option it may be possible to assembly an array in the wrong way and data can be incorrect, so it is better to be careful.
>
> Lukasz
>
>> On Mon, Jan 14, 2013 at 9:24 AM, Dorau, Lukasz <lukasz.dorau@intel.com>
>> wrote:
>> > On Monday, January 14, 2013 3:11 PM Dorau, Lukasz
>> <lukasz.dorau@intel.com> wrote:
>> >> On Monday, January 14, 2013 1:56 AM chris <tknchris@gmail.com> wrote:
>> >> > [292295.923942] bio: create slab <bio-1> at 1
>> >> > [292295.923965] md/raid:md126: not clean -- starting background
>> >> > reconstruction
>> >> > [292295.924000] md/raid:md126: device sdb operational as raid disk 2
>> >> > [292295.924005] md/raid:md126: device sdc operational as raid disk 1
>> >> > [292295.924009] md/raid:md126: device sde operational as raid disk 0
>> >> > [292295.925149] md/raid:md126: allocated 4250kB
>> >> > [292295.927268] md/raid:md126: cannot start dirty degraded array.
>> >>
>> >> Hi
>> >>
>> >> *Remember to backup the disks you have before trying the following! *
>> >>
>> >> You can try starting dirty degraded array using:
>> >> # mdadm --assemble --force ....
>> >>
>> >
>> > I meant adding --force option to:
>> > # mdadm --create --verbose --force /dev/md/Volume0 /dev/sdc missing
>> /dev/sdb /dev/sdd --raid-devices 4 --level=5
>> >
>> > Be very careful using "--force" option, because it can cause data corruption!
>> >
>> > Lukasz
>> >
>> >
>> >> See also the "Boot time assembly of degraded/dirty arrays" chapter in:
>> >> http://www.kernel.org/doc/Documentation/md.txt
>> >> (you can boot with option md-mod.start_dirty_degraded=1)
>> >>
>> >> Lukasz
>> >>
>> >>
>> >> > [292295.929666] RAID conf printout:
>> >> > [292295.929677] --- level:5 rd:4 wd:3
>> >> > [292295.929683] disk 0, o:1, dev:sde
>> >> > [292295.929688] disk 1, o:1, dev:sdc
>> >> > [292295.929693] disk 2, o:1, dev:sdb
>> >> > [292295.930898] md/raid:md126: failed to run raid set.
>> >> > [292295.930902] md: pers->run() failed ...
>> >> > [292295.931079] md: md126 stopped.
>> >> > [292295.931096] md: unbind<sdb>
>> >> > [292295.944228] md: export_rdev(sdb)
>> >> > [292295.944267] md: unbind<sdc>
>> >> > [292295.958126] md: export_rdev(sdc)
>> >> > [292295.958167] md: unbind<sde>
>> >> > [292295.970902] md: export_rdev(sde)
>> >> > [292296.219837] device-mapper: table: 252:1: raid45: unknown target type
>> >> > [292296.219845] device-mapper: ioctl: error adding target to table
>> >> > [292296.291542] device-mapper: table: 252:1: raid45: unknown target type
>> >> > [292296.291548] device-mapper: ioctl: error adding target to table
>> >> > [292296.310926] quiet_error: 1116 callbacks suppressed
>> >> > [292296.310934] Buffer I/O error on device dm-0, logical block
>> 3907022720
>> >> > [292296.310940] Buffer I/O error on device dm-0, logical block
>> 3907022721
>> >> > [292296.310944] Buffer I/O error on device dm-0, logical block
>> 3907022722
>> >> > [292296.310949] Buffer I/O error on device dm-0, logical block
>> 3907022723
>> >> > [292296.310953] Buffer I/O error on device dm-0, logical block
>> 3907022724
>> >> > [292296.310958] Buffer I/O error on device dm-0, logical block
>> 3907022725
>> >> > [292296.310962] Buffer I/O error on device dm-0, logical block
>> 3907022726
>> >> > [292296.310966] Buffer I/O error on device dm-0, logical block
>> 3907022727
>> >> > [292296.310973] Buffer I/O error on device dm-0, logical block
>> 3907022720
>> >> > [292296.310977] Buffer I/O error on device dm-0, logical block
>> 3907022721
>> >> > [292296.319968] device-mapper: table: 252:1: raid45: unknown target type
>> >> > [292296.319975] device-mapper: ioctl: error adding target to table
>> >> >
>> >> > Any ideas from here? Am I up the creek without a paddle? :(
>> >> >
>> >> > thanks to everyone for all your help so far
>> >> > chris
>> >> >
>> >> > On Sun, Jan 13, 2013 at 4:05 PM, Dan Williams <djbw@fb.com> wrote:
>> >> > >
>> >> > >
>> >> > > On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
>> >> > >
>> >> > >>Neil/Dave,
>> >> > >>
>> >> > >>Is it not possible to create imsm container with missing disk?
>> >> > >>If not, Is there any way to recreate the array with all disks but
>> >> > >>prevent any kind of sync which may overwrite array data?
>> >> > >
>> >> > > The example was in that link I sent:
>> >> > >
>> >> > > mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
>> >> > > mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l
>> 5
>> >> > >
>> >> > > The first command marks all devices as spares. The second creates the
>> >> > > degraded array.
>> >> > >
>> >> > > You probably want at least sdb and sdd in there since they have a copy of
>> >> > > the metadata.
>> >> > >
>> >> > > --
>> >> > > Dan
>> >> > >
>> >> > --
>> >> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> >> > the body of a message to majordomo@vger.kernel.org
>> >> > More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-16 16:49 ` chris
@ 2013-01-16 16:53 ` chris
2013-01-16 22:47 ` Dan Williams
1 sibling, 0 replies; 18+ messages in thread
From: chris @ 2013-01-16 16:53 UTC (permalink / raw)
To: Dorau, Lukasz; +Cc: Neil Brown, Jiang, Dave, Linux-RAID, Dan Williams
Just an idea... if I plugged disks into a machine with IMSM OROM,
could I "create" the array again inside the OROM and try to bring it
online without wiping any data? I'm not too sure if the OROM would let
me create a degraded array and I'm just wondering if that would do
anything different than what I'm trying with mdadm
chris
On Wed, Jan 16, 2013 at 11:49 AM, chris <tknchris@gmail.com> wrote:
> Hi, so after painfully imaging 4x2tb I have tried the suggestions of
> the other 2 permutations and adding --force with no change
>
> Other 2 permutations:
> # mdadm --stop /dev/md/imsm
> # export IMSM_NO_PLATFORM=1
> # mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
> /dev/sde --raid-devices 4 --metadata=imsm
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdb.
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdc.
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdd.
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sde.
> mdadm: size set to 1953511431K
> Continue creating array? y
> mdadm: container /dev/md/imsm prepared.
> # mdadm --create --verbose /dev/md/Volume0 /dev/sdc missing /dev/sdb
> /dev/sdd --raid-devices 4 --level=5
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: super1.x cannot open /dev/sdc: Device or resource busy
> mdadm: chunk size defaults to 128K
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdc but will be lost or
> meaningless after creating array
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdb but will be lost or
> meaningless after creating array
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdd but will be lost or
> meaningless after creating array
> mdadm: size set to 1953511424K
> Continue creating array? y
> mdadm: Creating array inside imsm container /dev/md/imsm
> mdadm: failed to activate array.
>
> # mdadm --stop /dev/md/imsm
> # export IMSM_NO_PLATFORM=1
> # mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
> /dev/sde --raid-devices 4 --metadata=imsm
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdb.
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdc.
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdd.
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sde.
> mdadm: size set to 1953511431K
> Continue creating array? y
> mdadm: container /dev/md/imsm prepared.
> # mdadm --create --verbose /dev/md/Volume0 missing /dev/sde /dev/sdb
> /dev/sdd --raid-devices 4 --level=5
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: super1.x cannot open /dev/sde: Device or resource busy
> mdadm: chunk size defaults to 128K
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sde but will be lost or
> meaningless after creating array
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdb but will be lost or
> meaningless after creating array
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdd but will be lost or
> meaningless after creating array
> mdadm: size set to 1953511424K
> Continue creating array? y
> mdadm: Creating array inside imsm container /dev/md/imsm
> mdadm: failed to activate array.
>
> Tried again with --force but same thing:
>
> # mdadm --stop /dev/md/imsm
> mdadm: stopped /dev/md/imsm
> # export IMSM_NO_PLATFORM=1
> # mdadm --create --verbose /dev/md/imsm /dev/sdb /dev/sdc /dev/sdd
> /dev/sde --raid-devices 4 --metadata=imsm
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdb.
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdc.
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sdd.
> mdadm: /dev/sde appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: metadata will over-write last partition on /dev/sde.
> mdadm: size set to 1953511431K
> Continue creating array? y
> mdadm: container /dev/md/imsm prepared.
> # mdadm --create --verbose --force /dev/md/Volume0 /dev/sdc missing
> /dev/sdb /dev/sdd --raid-devices 4 --level=5
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: super1.x cannot open /dev/sdc: Device or resource busy
> mdadm: chunk size defaults to 128K
> mdadm: /dev/sdc appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdc but will be lost or
> meaningless after creating array
> mdadm: /dev/sdb appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdb but will be lost or
> meaningless after creating array
> mdadm: /dev/sdd appears to be part of a raid array:
> level=container devices=0 ctime=Thu Jan 1 00:00:00 1970
> mdadm: partition table exists on /dev/sdd but will be lost or
> meaningless after creating array
> mdadm: size set to 1953511424K
> Continue creating array? y
> mdadm: Creating array inside imsm container /dev/md/imsm
> mdadm: failed to activate array.
>
> dmesg shows:
> [522342.240901] md: md127 stopped.
> [522342.240925] md: unbind<sde>
> [522342.251221] md: export_rdev(sde)
> [522342.251457] md: unbind<sdd>
> [522342.267808] md: export_rdev(sdd)
> [522342.267999] md: unbind<sdc>
> [522342.281160] md: export_rdev(sdc)
> [522342.281309] md: unbind<sdb>
> [522342.291136] md: export_rdev(sdb)
> [522351.758217] md: bind<sdb>
> [522351.758409] md: bind<sdc>
> [522351.758552] md: bind<sdd>
> [522351.758690] md: bind<sde>
> [522368.121090] md: bind<sdc>
> [522368.122401] md: bind<sdb>
> [522368.122577] md: bind<sdd>
> [522368.147454] bio: create slab <bio-1> at 1
> [522368.147477] md/raid:md126: not clean -- starting background reconstruction
> [522368.147515] md/raid:md126: device sdd operational as raid disk 3
> [522368.147520] md/raid:md126: device sdb operational as raid disk 2
> [522368.147525] md/raid:md126: device sdc operational as raid disk 0
> [522368.148651] md/raid:md126: allocated 4250kB
> [522368.152966] md/raid:md126: cannot start dirty degraded array.
> [522368.155245] RAID conf printout:
> [522368.155259] --- level:5 rd:4 wd:3
> [522368.155269] disk 0, o:1, dev:sdc
> [522368.155275] disk 2, o:1, dev:sdb
> [522368.155281] disk 3, o:1, dev:sdd
> [522368.157095] md/raid:md126: failed to run raid set.
> [522368.157102] md: pers->run() failed ...
> [522368.157418] md: md126 stopped.
> [522368.157435] md: unbind<sdd>
> [522368.167883] md: export_rdev(sdd)
> [522368.167922] md: unbind<sdb>
> [522368.181259] md: export_rdev(sdb)
> [522368.181302] md: unbind<sdc>
> [522368.194576] md: export_rdev(sdc)
> [522368.701814] device-mapper: table: 252:1: raid45: unknown target type
> [522368.701820] device-mapper: ioctl: error adding target to table
> [522368.775341] device-mapper: table: 252:1: raid45: unknown target type
> [522368.775347] device-mapper: ioctl: error adding target to table
> [522368.876314] quiet_error: 1116 callbacks suppressed
> [522368.876324] Buffer I/O error on device dm-0, logical block 3907022720
> [522368.876331] Buffer I/O error on device dm-0, logical block 3907022721
> [522368.876335] Buffer I/O error on device dm-0, logical block 3907022722
> [522368.876340] Buffer I/O error on device dm-0, logical block 3907022723
> [522368.876344] Buffer I/O error on device dm-0, logical block 3907022724
> [522368.876348] Buffer I/O error on device dm-0, logical block 3907022725
> [522368.876352] Buffer I/O error on device dm-0, logical block 3907022726
> [522368.876356] Buffer I/O error on device dm-0, logical block 3907022727
> [522368.876362] Buffer I/O error on device dm-0, logical block 3907022720
> [522368.876366] Buffer I/O error on device dm-0, logical block 3907022721
> [522368.883428] device-mapper: table: 252:1: raid45: unknown target type
> [522368.883434] device-mapper: ioctl: error adding target to table
> [522371.066343] device-mapper: table: 252:1: raid45: unknown target type
> [522371.066350] device-mapper: ioctl: error adding target to table
>
> any idea why it wont assemble? I thought even if data was corrupt I
> should be able to force it to assemble and look at it to determine if
> it is corrupt or in tact
>
> thanks
> chris
>
> On Tue, Jan 15, 2013 at 5:25 AM, Dorau, Lukasz <lukasz.dorau@intel.com> wrote:
>> On Monday, January 14, 2013 4:25 PM chris <tknchris@gmail.com> wrote:
>>> Ok thanks for the tips, I am imaging the disks now and will try after
>>> that is done. Just out of curiousity what could become corrupted by
>>> forcing the assemble? I was under the impression that as long as I
>>> have one member missing that the only thing that would be touched is
>>> metadata, is that right?
>>>
>> Yes, that is right. I meant that using --force option it may be possible to assembly an array in the wrong way and data can be incorrect, so it is better to be careful.
>>
>> Lukasz
>>
>>> On Mon, Jan 14, 2013 at 9:24 AM, Dorau, Lukasz <lukasz.dorau@intel.com>
>>> wrote:
>>> > On Monday, January 14, 2013 3:11 PM Dorau, Lukasz
>>> <lukasz.dorau@intel.com> wrote:
>>> >> On Monday, January 14, 2013 1:56 AM chris <tknchris@gmail.com> wrote:
>>> >> > [292295.923942] bio: create slab <bio-1> at 1
>>> >> > [292295.923965] md/raid:md126: not clean -- starting background
>>> >> > reconstruction
>>> >> > [292295.924000] md/raid:md126: device sdb operational as raid disk 2
>>> >> > [292295.924005] md/raid:md126: device sdc operational as raid disk 1
>>> >> > [292295.924009] md/raid:md126: device sde operational as raid disk 0
>>> >> > [292295.925149] md/raid:md126: allocated 4250kB
>>> >> > [292295.927268] md/raid:md126: cannot start dirty degraded array.
>>> >>
>>> >> Hi
>>> >>
>>> >> *Remember to backup the disks you have before trying the following! *
>>> >>
>>> >> You can try starting dirty degraded array using:
>>> >> # mdadm --assemble --force ....
>>> >>
>>> >
>>> > I meant adding --force option to:
>>> > # mdadm --create --verbose --force /dev/md/Volume0 /dev/sdc missing
>>> /dev/sdb /dev/sdd --raid-devices 4 --level=5
>>> >
>>> > Be very careful using "--force" option, because it can cause data corruption!
>>> >
>>> > Lukasz
>>> >
>>> >
>>> >> See also the "Boot time assembly of degraded/dirty arrays" chapter in:
>>> >> http://www.kernel.org/doc/Documentation/md.txt
>>> >> (you can boot with option md-mod.start_dirty_degraded=1)
>>> >>
>>> >> Lukasz
>>> >>
>>> >>
>>> >> > [292295.929666] RAID conf printout:
>>> >> > [292295.929677] --- level:5 rd:4 wd:3
>>> >> > [292295.929683] disk 0, o:1, dev:sde
>>> >> > [292295.929688] disk 1, o:1, dev:sdc
>>> >> > [292295.929693] disk 2, o:1, dev:sdb
>>> >> > [292295.930898] md/raid:md126: failed to run raid set.
>>> >> > [292295.930902] md: pers->run() failed ...
>>> >> > [292295.931079] md: md126 stopped.
>>> >> > [292295.931096] md: unbind<sdb>
>>> >> > [292295.944228] md: export_rdev(sdb)
>>> >> > [292295.944267] md: unbind<sdc>
>>> >> > [292295.958126] md: export_rdev(sdc)
>>> >> > [292295.958167] md: unbind<sde>
>>> >> > [292295.970902] md: export_rdev(sde)
>>> >> > [292296.219837] device-mapper: table: 252:1: raid45: unknown target type
>>> >> > [292296.219845] device-mapper: ioctl: error adding target to table
>>> >> > [292296.291542] device-mapper: table: 252:1: raid45: unknown target type
>>> >> > [292296.291548] device-mapper: ioctl: error adding target to table
>>> >> > [292296.310926] quiet_error: 1116 callbacks suppressed
>>> >> > [292296.310934] Buffer I/O error on device dm-0, logical block
>>> 3907022720
>>> >> > [292296.310940] Buffer I/O error on device dm-0, logical block
>>> 3907022721
>>> >> > [292296.310944] Buffer I/O error on device dm-0, logical block
>>> 3907022722
>>> >> > [292296.310949] Buffer I/O error on device dm-0, logical block
>>> 3907022723
>>> >> > [292296.310953] Buffer I/O error on device dm-0, logical block
>>> 3907022724
>>> >> > [292296.310958] Buffer I/O error on device dm-0, logical block
>>> 3907022725
>>> >> > [292296.310962] Buffer I/O error on device dm-0, logical block
>>> 3907022726
>>> >> > [292296.310966] Buffer I/O error on device dm-0, logical block
>>> 3907022727
>>> >> > [292296.310973] Buffer I/O error on device dm-0, logical block
>>> 3907022720
>>> >> > [292296.310977] Buffer I/O error on device dm-0, logical block
>>> 3907022721
>>> >> > [292296.319968] device-mapper: table: 252:1: raid45: unknown target type
>>> >> > [292296.319975] device-mapper: ioctl: error adding target to table
>>> >> >
>>> >> > Any ideas from here? Am I up the creek without a paddle? :(
>>> >> >
>>> >> > thanks to everyone for all your help so far
>>> >> > chris
>>> >> >
>>> >> > On Sun, Jan 13, 2013 at 4:05 PM, Dan Williams <djbw@fb.com> wrote:
>>> >> > >
>>> >> > >
>>> >> > > On 1/13/13 11:00 AM, "chris" <tknchris@gmail.com> wrote:
>>> >> > >
>>> >> > >>Neil/Dave,
>>> >> > >>
>>> >> > >>Is it not possible to create imsm container with missing disk?
>>> >> > >>If not, Is there any way to recreate the array with all disks but
>>> >> > >>prevent any kind of sync which may overwrite array data?
>>> >> > >
>>> >> > > The example was in that link I sent:
>>> >> > >
>>> >> > > mdadm --create /dev/md/imsm /dev/sd[bde] -e imsm
>>> >> > > mdadm --create /dev/md/vol0 /dev/sde missing /dev/sdb /dev/sdd -n 4 -l
>>> 5
>>> >> > >
>>> >> > > The first command marks all devices as spares. The second creates the
>>> >> > > degraded array.
>>> >> > >
>>> >> > > You probably want at least sdb and sdd in there since they have a copy of
>>> >> > > the metadata.
>>> >> > >
>>> >> > > --
>>> >> > > Dan
>>> >> > >
>>> >> > --
>>> >> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> >> > the body of a message to majordomo@vger.kernel.org
>>> >> > More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-16 16:49 ` chris
2013-01-16 16:53 ` chris
@ 2013-01-16 22:47 ` Dan Williams
2013-01-17 15:12 ` Charles Polisher
2013-01-17 16:07 ` chris
1 sibling, 2 replies; 18+ messages in thread
From: Dan Williams @ 2013-01-16 22:47 UTC (permalink / raw)
To: chris; +Cc: Dorau, Lukasz, Neil Brown, Jiang, Dave, Linux-RAID
On Wed, Jan 16, 2013 at 8:49 AM, chris <tknchris@gmail.com> wrote:
> any idea why it wont assemble? I thought even if data was corrupt I
> should be able to force it to assemble and look at it to determine if
> it is corrupt or in tact
A bug in mdadm was introduced shortly after the "missing" support was
added. Looks like you need 3.2.6.
--
Dan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-16 22:47 ` Dan Williams
@ 2013-01-17 15:12 ` Charles Polisher
2013-01-17 16:07 ` chris
1 sibling, 0 replies; 18+ messages in thread
From: Charles Polisher @ 2013-01-17 15:12 UTC (permalink / raw)
To: Dan Williams; +Cc: chris, Dorau, Lukasz, Neil Brown, Jiang, Dave, Linux-RAID
Dan Williams wrote:
> chris <tknchris@gmail.com> wrote:
> > any idea why it wont assemble? I thought even if data was corrupt I
> > should be able to force it to assemble and look at it to determine if
> > it is corrupt or in tact
>
> A bug in mdadm was introduced shortly after the "missing" support was
> added. Looks like you need 3.2.6.
Probably unrelated, but in case you were unaware, ICHxR chipsets
have snared me in the past with bugs that were later corrected
by firmware updates from Intel. If I remember correctly from
upthread, you were at one point attempting to / running this
array with such hardware. If this is still an option, you might
check your firmware version and the release notes for any more
recent release. This won't fix software RAID but it could
conceivably coax the ICH10R into assembling the array.
--
Charles
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Recovery/Access of imsm raid via mdadm?
2013-01-16 22:47 ` Dan Williams
2013-01-17 15:12 ` Charles Polisher
@ 2013-01-17 16:07 ` chris
1 sibling, 0 replies; 18+ messages in thread
From: chris @ 2013-01-17 16:07 UTC (permalink / raw)
To: Dan Williams; +Cc: Dorau, Lukasz, Neil Brown, Jiang, Dave, Linux-RAID
Dan,
You hit the nail on the head, mdadm 3.2.6 is able to assemble and get
the arrays online, now just going through the permutations to try to
get recoverable data but at this point recovery is normal and
non-intel specific.
Thanks to everyone who helped!
chris
On Wed, Jan 16, 2013 at 5:47 PM, Dan Williams <djbw@fb.com> wrote:
> On Wed, Jan 16, 2013 at 8:49 AM, chris <tknchris@gmail.com> wrote:
>> any idea why it wont assemble? I thought even if data was corrupt I
>> should be able to force it to assemble and look at it to determine if
>> it is corrupt or in tact
>
> A bug in mdadm was introduced shortly after the "missing" support was
> added. Looks like you need 3.2.6.
>
> --
> Dan
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2013-01-17 16:07 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-01-10 16:23 Recovery/Access of imsm raid via mdadm? chris
2013-01-10 17:09 ` Dave Jiang
2013-01-10 20:19 ` chris
2013-01-11 1:42 ` Dan Williams
2013-01-11 17:53 ` chris
2013-01-13 19:00 ` chris
2013-01-13 21:05 ` Dan Williams
2013-01-14 0:56 ` chris
2013-01-14 12:36 ` Dorau, Lukasz
2013-01-14 14:10 ` Dorau, Lukasz
2013-01-14 14:24 ` Dorau, Lukasz
2013-01-14 15:25 ` chris
2013-01-15 10:25 ` Dorau, Lukasz
2013-01-16 16:49 ` chris
2013-01-16 16:53 ` chris
2013-01-16 22:47 ` Dan Williams
2013-01-17 15:12 ` Charles Polisher
2013-01-17 16:07 ` chris
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).