From: NeilBrown <neilb@suse.de>
To: Christian <cjs703@gmail.com>
Cc: linux-raid@vger.kernel.org
Subject: Re: RAID6 array won't start
Date: Sat, 12 Feb 2011 07:49:04 +1100 [thread overview]
Message-ID: <20110212074904.41e113ed@notabene.brown> (raw)
In-Reply-To: <AANLkTi=dtMbuszU3x_hGduN-y5ATbgD0tgby_2mj4YsH@mail.gmail.com>
On Fri, 11 Feb 2011 11:48:41 -0500 Christian <cjs703@gmail.com> wrote:
> Hello,
>
> I have a 10 disk RAID6 array. I had an issue with a molex connector
> that caused drives to drop from a 5in3 SATA backplane. There wasn't
> any activity occurring on the array during the time, so I think the
> filesystem should be in tact. The issue now is the event counts are
> out of sync, along with the state of each drive.
>
> # mdadm -A /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> mdadm: /dev/md1 assembled from 8 drives - not enough to start the
> array while not clean - consider --force.
>
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md1 : inactive sdt1[1](S) sde1[9](S) sdl1[8](S) sdg1[7](S) sdf1[6](S)
> sdk1[5](S) sdq1[4](S) sds1[3](S) sdr1[2](S) sdm1[0](S)
> 19535134720 blocks
>
>
> # mdadm -Af /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> mdadm: failed to RUN_ARRAY /dev/md1: Input/output error
The array is already in a partially-assembled state.
mdadm -S /dev/md1
before trying to assemble again with -f.
NeilBrown
>
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md1 : inactive sdt1[1] sde1[9] sdl1[8] sdg1[7] sdf1[6] sdk1[5] sdq1[4] sdr1[2]
> 15628107776 blocks
>
> I've read others having success with this but it's not accepting it.
> # echo "clean" > /sys/block/md1/md/array_state
> bash: echo: write error: Invalid argument
>
>
> # mdadm -E /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1 /dev/sdm1
> /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1 | egrep Event
> Events : 147226
> Events : 147226
> Events : 147226
> Events : 147226
> Events : 147226
> Events : 147221
> Events : 147225
> Events : 147225
> Events : 147221
> Events : 147225
>
>
> # mdadm -E /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1 /dev/sdm1
> /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7ead3 - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 9 8 65 9 active sync /dev/sde1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdf1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7eadd - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 6 8 81 6 active sync /dev/sdf1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdg1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7eaef - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 7 8 97 7 active sync /dev/sdg1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdk1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7eb2b - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 5 8 161 5 active sync /dev/sdk1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdl1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:24:09 2011
> State : active
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27a7eb41 - correct
> Events : 147226
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 8 8 177 8 active sync /dev/sdl1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdm1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 10
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 18:45:59 2011
> State : clean
> Active Devices : 10
> Working Devices : 10
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa228c - correct
> Events : 147221
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 0 8 193 0 active sync /dev/sdm1
>
> 0 0 8 193 0 active sync /dev/sdm1
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 65 33 3 active sync /dev/sds1
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdq1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:20:33 2011
> State : clean
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa2903 - correct
> Events : 147225
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 4 65 1 4 active sync /dev/sdq1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdr1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:20:33 2011
> State : clean
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa290f - correct
> Events : 147225
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 2 65 17 2 active sync /dev/sdr1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sds1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 10
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 18:45:59 2011
> State : clean
> Active Devices : 10
> Working Devices : 10
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa222b - correct
> Events : 147221
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 3 65 33 3 active sync /dev/sds1
>
> 0 0 8 193 0 active sync /dev/sdm1
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 65 33 3 active sync /dev/sds1
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
> /dev/sdt1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
> Creation Time : Sat Sep 25 13:23:40 2010
> Raid Level : raid6
> Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
> Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
> Raid Devices : 10
> Total Devices : 8
> Preferred Minor : 1
>
> Update Time : Wed Feb 9 19:20:33 2011
> State : clean
> Active Devices : 8
> Working Devices : 8
> Failed Devices : 0
> Spare Devices : 0
> Checksum : 27aa292d - correct
> Events : 147225
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Number Major Minor RaidDevice State
> this 1 65 49 1 active sync /dev/sdt1
>
> 0 0 0 0 0 removed
> 1 1 65 49 1 active sync /dev/sdt1
> 2 2 65 17 2 active sync /dev/sdr1
> 3 3 0 0 3 faulty removed
> 4 4 65 1 4 active sync /dev/sdq1
> 5 5 8 161 5 active sync /dev/sdk1
> 6 6 8 81 6 active sync /dev/sdf1
> 7 7 8 97 7 active sync /dev/sdg1
> 8 8 8 177 8 active sync /dev/sdl1
> 9 9 8 65 9 active sync /dev/sde1
>
>
> I've read others who have had success or failure using create to get
> things back in order again. This is what I came up with the drives
> listed in sequence from 0 to 9. The drives marked missing are the two
> that have event counts much lower than the rest.
>
> # mdadm --create /dev/md1 --verbose --level=6 --raid-devices=10
> --chunk=64 missing /dev/sdt1 /dev/sdr1 missing /dev/sdq1 /dev/sdk1
> /dev/sdf1 /dev/sdg1 /dev/sdl1 /dev/sde1
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdt1 appears to contain an ext2fs file system
> size=-1551761408K mtime=Wed Feb 9 18:45:59 2011
> mdadm: /dev/sdt1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdr1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdq1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdk1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdf1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdg1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdl1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sde1 appears to contain an ext2fs file system
> size=-591265728K mtime=Mon Feb 14 13:17:27 2028
> mdadm: /dev/sde1 appears to be part of a raid array:
> level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: size set to 1953512384K
> Continue creating array? n
>
> I haven't executed this yet though.
>
>
> Any help would be appreciated,
> Thanks
> N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±¨_Š{ayº\x1dʇڙë,j\a¢f£¢·hš‹àz¹\x1e®w¥¢¸\f¢·¦j:+v‰¨ŠwèjØm¶Ÿÿ¾\a«‘êçzZ+ƒùšŽŠÝ¢j"_ú!
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2011-02-11 20:49 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <AANLkTimLfJHc2waPkvjUXn=mewc37qkCd-r0GG4k=e-D@mail.gmail.com>
2011-02-11 16:48 ` RAID6 array won't start Christian
2011-02-11 20:49 ` NeilBrown [this message]
2011-02-11 21:13 ` Christian
2011-02-11 21:32 ` NeilBrown
2011-02-11 21:51 ` Christian
2011-02-11 22:36 ` Christian
2011-02-11 23:37 ` Christian
2011-02-12 0:52 ` NeilBrown
2011-02-12 0:57 ` Christian
2011-02-12 1:03 ` NeilBrown
2011-02-12 4:08 ` Christian
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110212074904.41e113ed@notabene.brown \
--to=neilb@suse.de \
--cc=cjs703@gmail.com \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.