From: Maarten <maarten@ultratux.net>
To: linux-raid@vger.kernel.org
Subject: re-assembling arrays after severe controller hardware issues...
Date: Fri, 25 Oct 2013 23:19:21 +0200 [thread overview]
Message-ID: <526AE059.8020006@ultratux.net> (raw)
Hi
I suffered a coupleof (subsequent) multi-disk raid failures caused by
hardware (not disk) issues. I now am reasonably confident the cause of
the hardware issues has been remedied. Looking at one of the arrays I
see differing event counters, but all close together. If I attempt
--assemble that doesn't work, mdadm insists on only taking two of the
six raid6 members into account.. Am I correct that an added --force will
rectify that, or is a different course of action necessary ?
mouse ~ # cat /proc/mdstat
md1 : inactive sdi1[0](S) sdh1[5](S) sdb1[4](S) sdg1[2](S) sdc1[1](S)
4869638272 blocks
mouse ~ # mdadm --examine /dev/sd[ihgcb]1|grep -i uuid
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
mouse ~ # mdadm --examine /dev/sd[ihgcb]1|grep -i events
Events : 115315
Events : 115308
Events : 115312
Events : 115311
Events : 115315
It looks like it just takes the two 115315 ones, I figure. I need to
tell it to consider all five, regardless of events counter. Is
--assemble --force the right thing here?
(Yes, there was a sixth member too, but I suffered a crash (kernel
kicked a controller with 5 disks on it) during the re-adding of that one
disk, so I have therefore deemed it unsafe to use for --assemble, so
that one is marked and put away safely)
cheers,
Maarten
For the sake of completeness, here's the full --examine outputs:
/dev/sdb1:
Magic : a92b4efc
Version : 0.90.00
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
Creation Time : Sun Jan 31 23:07:37 2010
Raid Level : raid6
Used Dev Size : 971932352 (926.91 GiB 995.26 GB)
Array Size : 3887729408 (3707.63 GiB 3981.03 GB)
Raid Devices : 6
Total Devices : 5
Preferred Minor : 1
Update Time : Fri Sep 27 22:56:30 2013
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 4
Spare Devices : 0
Checksum : 5d28eaa8 - correct
Events : 115315
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 4 8 17 4 active sync /dev/sdb1
0 0 8 209 0 active sync
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 0 0 3 faulty removed
4 4 8 17 4 active sync /dev/sdb1
5 5 0 0 5 faulty removed
/dev/sdc1:
Magic : a92b4efc
Version : 0.90.00
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
Creation Time : Sun Jan 31 23:07:37 2010
Raid Level : raid6
Used Dev Size : 971932352 (926.91 GiB 995.26 GB)
Array Size : 3887729408 (3707.63 GiB 3981.03 GB)
Raid Devices : 6
Total Devices : 6
Preferred Minor : 1
Update Time : Wed Sep 25 12:42:47 2013
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0
Checksum : 5d25b882 - correct
Events : 115308
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 1 8 129 1 active sync /dev/sdi1
0 0 8 209 0 active sync
1 1 8 129 1 active sync /dev/sdi1
2 2 8 65 2 active sync /dev/sde1
3 3 8 113 3 active sync /dev/sdh1
4 4 8 17 4 active sync /dev/sdb1
5 5 8 81 5 active sync /dev/sdf1
/dev/sdg1:
Magic : a92b4efc
Version : 0.90.00
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
Creation Time : Sun Jan 31 23:07:37 2010
Raid Level : raid6
Used Dev Size : 971932352 (926.91 GiB 995.26 GB)
Array Size : 3887729408 (3707.63 GiB 3981.03 GB)
Raid Devices : 6
Total Devices : 5
Preferred Minor : 1
Update Time : Fri Sep 27 22:40:25 2013
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 3
Spare Devices : 1
Checksum : 5d28e6f7 - correct
Events : 115312
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 2 8 65 2 active sync /dev/sde1
0 0 8 209 0 active sync
1 1 0 0 1 faulty removed
2 2 8 65 2 active sync /dev/sde1
3 3 0 0 3 faulty removed
4 4 8 17 4 active sync /dev/sdb1
5 5 0 0 5 faulty removed
6 6 8 113 6 spare /dev/sdh1
/dev/sdh1:
Magic : a92b4efc
Version : 0.90.00
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
Creation Time : Sun Jan 31 23:07:37 2010
Raid Level : raid6
Used Dev Size : 971932352 (926.91 GiB 995.26 GB)
Array Size : 3887729408 (3707.63 GiB 3981.03 GB)
Raid Devices : 6
Total Devices : 5
Preferred Minor : 1
Update Time : Fri Sep 27 22:23:09 2013
State : clean
Active Devices : 4
Working Devices : 5
Failed Devices : 2
Spare Devices : 1
Checksum : 5d28e2ee - correct
Events : 115311
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 5 8 81 5 active sync /dev/sdf1
0 0 8 209 0 active sync
1 1 0 0 1 faulty removed
2 2 8 65 2 active sync /dev/sde1
3 3 0 0 3 faulty removed
4 4 8 17 4 active sync /dev/sdb1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 113 6 spare /dev/sdh1
/dev/sdi1:
Magic : a92b4efc
Version : 0.90.00
UUID : e1b1c874:1000f29b:d02b8c28:1a80597f
Creation Time : Sun Jan 31 23:07:37 2010
Raid Level : raid6
Used Dev Size : 971932352 (926.91 GiB 995.26 GB)
Array Size : 3887729408 (3707.63 GiB 3981.03 GB)
Raid Devices : 6
Total Devices : 5
Preferred Minor : 1
Update Time : Fri Sep 27 22:56:30 2013
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 4
Spare Devices : 0
Checksum : 5d28eb60 - correct
Events : 115315
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 0 8 209 0 active sync
0 0 8 209 0 active sync
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 0 0 3 faulty removed
4 4 8 17 4 active sync /dev/sdb1
5 5 0 0 5 faulty removed
next reply other threads:[~2013-10-25 21:19 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-10-25 21:19 Maarten [this message]
2013-10-25 23:06 ` re-assembling arrays after severe controller hardware issues Phil Turmel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=526AE059.8020006@ultratux.net \
--to=maarten@ultratux.net \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.