From: Chris Allen <chris@cjx.com>
To: Neil Brown <neilb@suse.de>
Cc: linux-raid@vger.kernel.org
Subject: Re: Recovering a raid5 array with strange event count
Date: Fri, 13 Apr 2007 14:07:58 +0100 [thread overview]
Message-ID: <461F80AE.5080702@cjx.com> (raw)
In-Reply-To: <17951.29662.910442.896659@notabene.brown>
Neil Brown wrote:
> On Friday April 13, chris@cjx.com wrote:
>
>> Dear All,
>>
>> I have an 8-drive raid-5 array running under 2.6.11. This morning it
>> bombed out, and when I brought
>> it up again, two drives had incorrect event counts:
>>
>>
>> sda1: 0.8258715
>> sdb1: 0.8258715
>> sdc1: 0.8258715
>> sdd1: 0.8258715
>> sde1: 0.8258715
>> sdf1: 0.8258715
>> sdg1: 0.8258708
>> sdh1: 0.8258716
>>
>>
>> sdg1 is out of date (expected), but sdh1 has received an extra event.
>>
>> Any attempt to restart with mdadm --assemble --force, results in an an
>> un-startable array with an event count of 0.8258715.
>>
>> Can anybody advise on the correct command to use to get it started again?
>> I'm assuming I'll need to use mdadm --create --assume-clean - but I'm
>> not sure
>> which drives should be included/excluded when I do this.
>>
>
> A difference of 1 in event counts is not supposed to cause a problem.
> Have you tried simply assembling the array without including sdg1.
> e.g.
> mdadm -A /dev/md0 /dev/sd[abcdefh]1
>
>
>
# mdadm -A /dev/md0 /dev/sd[abcdefh]1
mdadm: /dev/md0 assembled from 7 drives - need all 8 to start it (use
--run to insist)
# mdadm -D /dev/md0
mdadm: md device /dev/md0 does not appear to be active.
mdadm --run /dev/md0
mdadm: failed to run array /dev/md0: invalid argument
I've attached the syslog, the dump for the assembled array, the dump for
each drive
and the contents of /proc/mdstat. Using --force makes no difference.
Apr 13 13:59:45 snap29 kernel: md: bind<sdb1>
Apr 13 13:59:45 snap29 kernel: md: bind<sdc1>
Apr 13 13:59:45 snap29 kernel: md: bind<sdd1>
Apr 13 13:59:45 snap29 kernel: md: bind<sde1>
Apr 13 13:59:45 snap29 kernel: md: bind<sdf1>
Apr 13 13:59:45 snap29 kernel: md: bind<sdh1>
Apr 13 13:59:45 snap29 kernel: md: bind<sda1>
Apr 13 14:00:01 snap29 kernel: md: md0: raid array is not clean --
starting background reconstruction
Apr 13 14:00:01 snap29 kernel: raid5: device sda1 operational as raid disk 0
Apr 13 14:00:01 snap29 kernel: raid5: device sdh1 operational as raid disk 7
Apr 13 14:00:01 snap29 kernel: raid5: device sdf1 operational as raid disk 5
Apr 13 14:00:01 snap29 kernel: raid5: device sde1 operational as raid disk 4
Apr 13 14:00:01 snap29 kernel: raid5: device sdd1 operational as raid disk 3
Apr 13 14:00:01 snap29 kernel: raid5: device sdc1 operational as raid disk 2
Apr 13 14:00:01 snap29 kernel: raid5: device sdb1 operational as raid disk 1
Apr 13 14:00:01 snap29 kernel: raid5: cannot start dirty degraded array
for md0
Apr 13 14:00:01 snap29 kernel: RAID5 conf printout:
Apr 13 14:00:01 snap29 kernel: --- rd:8 wd:7 fd:1
Apr 13 14:00:01 snap29 kernel: disk 0, o:1, dev:sda1
Apr 13 14:00:01 snap29 kernel: disk 1, o:1, dev:sdb1
Apr 13 14:00:01 snap29 kernel: disk 2, o:1, dev:sdc1
Apr 13 14:00:01 snap29 kernel: disk 3, o:1, dev:sdd1
Apr 13 14:00:01 snap29 kernel: disk 4, o:1, dev:sde1
Apr 13 14:00:01 snap29 kernel: disk 5, o:1, dev:sdf1
Apr 13 14:00:01 snap29 kernel: disk 7, o:1, dev:sdh1
Apr 13 14:00:01 snap29 kernel: raid5: failed to run raid set md0
Apr 13 14:00:01 snap29 kernel: md: pers->run() failed ...
/dev/md0:
Version : 00.90.01
Creation Time : Wed Apr 19 06:23:21 2006
Raid Level : raid5
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Raid Devices : 8
Total Devices : 7
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Fri Apr 13 10:11:15 2007
State : active, degraded, Not Started
Active Devices : 7
Working Devices : 7
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
UUID : 31b253f9:02049908:aa4bb1ab:753b8fda
Events : 0.8258715
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
2 8 33 2 active sync /dev/sdc1
3 8 49 3 active sync /dev/sdd1
4 8 65 4 active sync /dev/sde1
5 8 81 5 active sync /dev/sdf1
6 0 0 6 removed
7 8 113 7 active sync /dev/sdh1
/dev/sda1:
Magic : a92b4efc
Version : 00.90.01
UUID : 31b253f9:02049908:aa4bb1ab:753b8fda
Creation Time : Wed Apr 19 06:23:21 2006
Raid Level : raid5
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Array Size : 3418687552 (3260.31 GiB 3500.74 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0
Update Time : Fri Apr 13 10:11:12 2007
State : clean
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Checksum : a469bd5a - correct
Events : 0.8258715
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 0 8 1 0 active sync /dev/sda1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 33 2 active sync /dev/sdc1
3 3 8 49 3 active sync /dev/sdd1
4 4 8 65 4 active sync /dev/sde1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 97 6 active sync /dev/sdg1
7 7 8 113 7 active sync /dev/sdh1
/dev/sdb1:
Magic : a92b4efc
Version : 00.90.01
UUID : 31b253f9:02049908:aa4bb1ab:753b8fda
Creation Time : Wed Apr 19 06:23:21 2006
Raid Level : raid5
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Array Size : 3418687552 (3260.31 GiB 3500.74 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0
Update Time : Fri Apr 13 10:11:12 2007
State : active
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Checksum : a469bd6b - correct
Events : 0.8258715
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 1 8 17 1 active sync /dev/sdb1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 33 2 active sync /dev/sdc1
3 3 8 49 3 active sync /dev/sdd1
4 4 8 65 4 active sync /dev/sde1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 97 6 active sync /dev/sdg1
7 7 8 113 7 active sync /dev/sdh1
/dev/sdc1:
Magic : a92b4efc
Version : 00.90.01
UUID : 31b253f9:02049908:aa4bb1ab:753b8fda
Creation Time : Wed Apr 19 06:23:21 2006
Raid Level : raid5
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Array Size : 3418687552 (3260.31 GiB 3500.74 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0
Update Time : Fri Apr 13 10:11:12 2007
State : active
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Checksum : a469bd7d - correct
Events : 0.8258715
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 2 8 33 2 active sync /dev/sdc1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 33 2 active sync /dev/sdc1
3 3 8 49 3 active sync /dev/sdd1
4 4 8 65 4 active sync /dev/sde1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 97 6 active sync /dev/sdg1
7 7 8 113 7 active sync /dev/sdh1
/dev/sdd1:
Magic : a92b4efc
Version : 00.90.01
UUID : 31b253f9:02049908:aa4bb1ab:753b8fda
Creation Time : Wed Apr 19 06:23:21 2006
Raid Level : raid5
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Array Size : 3418687552 (3260.31 GiB 3500.74 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0
Update Time : Fri Apr 13 10:11:12 2007
State : active
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Checksum : a469bd8f - correct
Events : 0.8258715
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 3 8 49 3 active sync /dev/sdd1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 33 2 active sync /dev/sdc1
3 3 8 49 3 active sync /dev/sdd1
4 4 8 65 4 active sync /dev/sde1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 97 6 active sync /dev/sdg1
7 7 8 113 7 active sync /dev/sdh1
/dev/sde1:
Magic : a92b4efc
Version : 00.90.01
UUID : 31b253f9:02049908:aa4bb1ab:753b8fda
Creation Time : Wed Apr 19 06:23:21 2006
Raid Level : raid5
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Array Size : 3418687552 (3260.31 GiB 3500.74 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0
Update Time : Fri Apr 13 10:11:12 2007
State : active
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Checksum : a469bda1 - correct
Events : 0.8258715
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 4 8 65 4 active sync /dev/sde1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 33 2 active sync /dev/sdc1
3 3 8 49 3 active sync /dev/sdd1
4 4 8 65 4 active sync /dev/sde1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 97 6 active sync /dev/sdg1
7 7 8 113 7 active sync /dev/sdh1
/dev/sdf1:
Magic : a92b4efc
Version : 00.90.01
UUID : 31b253f9:02049908:aa4bb1ab:753b8fda
Creation Time : Wed Apr 19 06:23:21 2006
Raid Level : raid5
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Array Size : 3418687552 (3260.31 GiB 3500.74 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0
Update Time : Fri Apr 13 10:11:12 2007
State : active
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Checksum : a469bdb3 - correct
Events : 0.8258715
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 5 8 81 5 active sync /dev/sdf1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 33 2 active sync /dev/sdc1
3 3 8 49 3 active sync /dev/sdd1
4 4 8 65 4 active sync /dev/sde1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 97 6 active sync /dev/sdg1
7 7 8 113 7 active sync /dev/sdh1
/dev/sdh1:
Magic : a92b4efc
Version : 00.90.01
UUID : 31b253f9:02049908:aa4bb1ab:753b8fda
Creation Time : Wed Apr 19 06:23:21 2006
Raid Level : raid5
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Array Size : 3418687552 (3260.31 GiB 3500.74 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0
Update Time : Fri Apr 13 10:11:15 2007
State : active
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Checksum : a469bddb - correct
Events : 0.8258716
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 7 8 113 7 active sync /dev/sdh1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 33 2 active sync /dev/sdc1
3 3 8 49 3 active sync /dev/sdd1
4 4 8 65 4 active sync /dev/sde1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 97 6 active sync /dev/sdg1
7 7 8 113 7 active sync /dev/sdh1
Personalities : [raid5]
md0 : inactive sda1[0] sdh1[7] sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1]
3418687552 blocks
unused devices: <none>
next prev parent reply other threads:[~2007-04-13 13:07 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-04-13 10:14 Recovering a raid5 array with strange event count Chris Allen
2007-04-13 12:13 ` Neil Brown
2007-04-13 13:07 ` Chris Allen [this message]
2007-04-16 13:55 ` Chris Allen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=461F80AE.5080702@cjx.com \
--to=chris@cjx.com \
--cc=linux-raid@vger.kernel.org \
--cc=neilb@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.