From: NeilBrown <neilb@suse.de>
To: Christian <cjs703@gmail.com>
Cc: linux-raid@vger.kernel.org
Subject: Re: RAID6 array won't start
Date: Sat, 12 Feb 2011 07:49:04 +1100 [thread overview]
Message-ID: <20110212074904.41e113ed@notabene.brown> (raw)
In-Reply-To: <AANLkTi=dtMbuszU3x_hGduN-y5ATbgD0tgby_2mj4YsH@mail.gmail.com>
On Fri, 11 Feb 2011 11:48:41 -0500 Christian <cjs703@gmail.com> wrote:
> Hello,
>
> I have a 10 disk RAID6 array. I had an issue with a molex connector
> that caused drives to drop from a 5in3 SATA backplane. There wasn't
> any activity occurring on the array during the time, so I think the
> filesystem should be in tact. The issue now is the event counts are
> out of sync, along with the state of each drive.
>
> # mdadm -A /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> mdadm: /dev/md1 assembled from 8 drives - not enough to start the
> array while not clean - consider --force.
>
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md1 : inactive sdt1[1](S) sde1[9](S) sdl1[8](S) sdg1[7](S) sdf1[6](S)
> sdk1[5](S) sdq1[4](S) sds1[3](S) sdr1[2](S) sdm1[0](S)
> 19535134720 blocks
>
>
> # mdadm -Af /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> mdadm: failed to RUN_ARRAY /dev/md1: Input/output error
The array is already in a partially-assembled state.
mdadm -S /dev/md1
before trying to assemble again with -f.
NeilBrown
>
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md1 : inactive sdt1[1] sde1[9] sdl1[8] sdg1[7] sdf1[6] sdk1[5] sdq1[4] sdr1[2]
> 15628107776 blocks
>
> I've read others having success with this but it's not accepting it.
> # echo "clean" > /sys/block/md1/md/array_state
> bash: echo: write error: Invalid argument
>
>
> # mdadm -E /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1 /dev/sdm1
> /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1 | egrep Event
> Events : 147226
> Events : 147226
> Events : 147226
> Events : 147226
> Events : 147226
> Events : 147221
> Events : 147225
> Events : 147225
> Events : 147221
> Events : 147225
>
>
> # mdadm -E /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1 /dev/sdm1
> /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7ead3 - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 9 8 65 9 active sync /dev/sde1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdf1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7eadd - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 6 8 81 6 active sync /dev/sdf1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdg1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7eaef - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 7 8 97 7 active sync /dev/sdg1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdk1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7eb2b - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 5 8 161 5 active sync /dev/sdk1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdl1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7eb41 - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 8 8 177 8 active sync /dev/sdl1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdm1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 10
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 18:45:59 2011
> State : clean
> Active Devices : 10
> Working Devices : 10
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa228c - correct
> Events : 147221
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 0 8 193 0 active sync /dev/sdm1
>
> 0 0 8 193 0 active sync /dev/sdm1
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 65 33 3 active sync /dev/sds1
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdq1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:20:33 2011
> State : clean
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa2903 - correct
> Events : 147225
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 4 65 1 4 active sync /dev/sdq1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdr1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:20:33 2011
> State : clean
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa290f - correct
> Events : 147225
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 2 65 17 2 active sync /dev/sdr1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sds1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 10
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 18:45:59 2011
> State : clean
> Active Devices : 10
> Working Devices : 10
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa222b - correct
> Events : 147221
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 3 65 33 3 active sync /dev/sds1
>
> 0 0 8 193 0 active sync /dev/sdm1
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 65 33 3 active sync /dev/sds1
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdt1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:20:33 2011
> State : clean
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa292d - correct
> Events : 147225
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 1 65 49 1 active sync /dev/sdt1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
>
>
> I've read others who have had success or failure using create to get
> things back in order again. This is what I came up with the drives
> listed in sequence from 0 to 9. The drives marked missing are the two
> that have event counts much lower than the rest.
>
> # mdadm --create /dev/md1 --verbose --level=6 --raid-devices=10
> --chunk=64 missing /dev/sdt1 /dev/sdr1 missing /dev/sdq1 /dev/sdk1
> /dev/sdf1 /dev/sdg1 /dev/sdl1 /dev/sde1
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdt1 appears to contain an ext2fs file system
> size=-1551761408K mtime=Wed Feb 9 18:45:59 2011
> mdadm: /dev/sdt1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdr1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdq1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdk1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdf1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdg1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdl1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sde1 appears to contain an ext2fs file system
> size=-591265728K mtime=Mon Feb 14 13:17:27 2028
> mdadm: /dev/sde1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: size set to 1953512384K
> Continue creating array? n
>
> I haven't executed this yet though.
>
>
> Any help would be appreciated,
> Thanks
> N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±¨_Š{ayº\x1dʇڙë,j\a¢f£¢·hš‹àz¹\x1e®w¥¢¸\f¢·¦j:+v‰¨ŠwèjØm¶Ÿÿ¾\a«‘êçzZ+ƒùšŽŠÝ¢j"_ú!
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2011-02-11 20:49 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <AANLkTimLfJHc2waPkvjUXn=mewc37qkCd-r0GG4k=e-D@mail.gmail.com>
2011-02-11 16:48 ` RAID6 array won't start Christian
2011-02-11 20:49 ` NeilBrown [this message]
2011-02-11 21:13 ` Christian
2011-02-11 21:32 ` NeilBrown
2011-02-11 21:51 ` Christian
2011-02-11 22:36 ` Christian
2011-02-11 23:37 ` Christian
2011-02-12 0:52 ` NeilBrown
2011-02-12 0:57 ` Christian
2011-02-12 1:03 ` NeilBrown
2011-02-12 4:08 ` Christian
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110212074904.41e113ed@notabene.brown \
--to=neilb@suse.de \
--cc=cjs703@gmail.com \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox