From: dujun <dujun@perabytes.com>
To: Neil Brown <neilb@suse.de>
Cc: "Mathias Burén" <mathias.buren@gmail.com>, linux-raid@vger.kernel.org
Subject: Re: raid related kernel hang in 2.6.36
Date: Thu, 25 Nov 2010 11:12:43 +0800 [thread overview]
Message-ID: <1290654763.5416.31.camel@dujun.perabytes.com> (raw)
In-Reply-To: <20101125070155.277a0d62@notabene.brown>
[-- Attachment #1: Type: text/plain, Size: 6586 bytes --]
Hi, Neil,
attached is the mdadm -E and mdadm -D information for md10(raid5 with 16
disks) and md11 (raid6 with 16 disks).
I recompiled the 2.6.36 kernel with kernel sysrq and softlock dectet
option, then setup two mds to reproduce the softlock bug.
First create two md using mdadm:
mdadm -C /dev/md10 -l 5 -n 16 /dev/sd[b-q]
mdadm -C /dev/md10 -l 6 -n 16 /dev/sd[r-z] /dev/sda[a-g]
then use dd :
dd if=/dev/zero of=/dev/md10 bs=1M &
dd if=/dev/zero of=/dev/md11 bs=1M&
After a while, the system is hung with lots of softlock kernel trace on
screen. following are some netconsole output:
Nov 25 11:01:35 10.10.20.20 [ 1236.027001] BUG: soft lockup - CPU#0
stuck for 67s! [top:5690]
Nov 25 11:01:40 10.10.20.20 [ 1236.027001] Stack:
Nov 25 11:01:45 10.10.20.20 [ 1236.027001] Call Trace:
Nov 25 11:01:50 10.10.20.20 [ 1236.027001] Code:
Nov 25 11:01:55 10.10.20.20 03
Nov 25 11:02:00 10.10.20.20 81
Nov 25 11:02:05 10.10.20.20 48
Nov 25 11:02:10 10.10.20.20 c7
Nov 25 11:02:15 10.10.20.20 c1
Nov 25 11:02:20 10.10.20.20 91
Nov 25 11:02:25 10.10.20.20 19
Nov 25 11:02:30 10.10.20.20 03
Nov 25 11:02:35 10.10.20.20 81
Nov 25 11:02:40 10.10.20.20 e9
Nov 25 11:02:45 10.10.20.20 dd
Nov 25 11:02:50 10.10.20.20 fe
Nov 25 11:02:55 10.10.20.20 ff
Nov 25 11:03:00 10.10.20.20 ff
Nov 25 11:03:05 10.10.20.20 90
Nov 25 11:03:10 10.10.20.20 90
Nov 25 11:03:15 10.10.20.20 90
Nov 25 11:03:20 10.10.20.20 90
Nov 25 11:03:25 10.10.20.20 90
Nov 25 11:03:30 10.10.20.20 90
Nov 25 11:03:35 10.10.20.20 90
Nov 25 11:03:40 10.10.20.20 90
Nov 25 11:03:45 10.10.20.20 90
Nov 25 11:03:50 10.10.20.20 90
Nov 25 11:03:55 10.10.20.20 90
Nov 25 11:04:00 10.10.20.20 90
Nov 25 11:04:05 10.10.20.20 90
Nov 25 11:04:10 10.10.20.20 b8
Nov 25 11:04:15 10.10.20.20 00
Nov 25 11:04:20 10.10.20.20 01
Nov 25 11:04:25 10.10.20.20 00
Nov 25 11:04:30 10.10.20.20 00
Nov 25 11:04:35 10.10.20.20 f0
Nov 25 11:04:40 10.10.20.20 66
Nov 25 11:04:45 10.10.20.20 0f
Nov 25 11:04:50 10.10.20.20 c1
Nov 25 11:04:55 10.10.20.20 07
Nov 25 11:05:00 10.10.20.20 38
Nov 25 11:05:05 10.10.20.20 e0
Nov 25 11:05:10 10.10.20.20 74
Nov 25 11:05:15 10.10.20.20 06
Nov 25 11:05:20 10.10.20.20 f3
Nov 25 11:05:25 10.10.20.20 90
Message from syslogd@10.10.20.20 at Nov 25 11:05:30 ...
a>
Nov 25 11:05:35 10.10.20.20 07
Nov 25 11:05:40 10.10.20.20 eb
Nov 25 11:05:45 10.10.20.20 f6
Nov 25 11:05:50 10.10.20.20 c3
Nov 25 11:05:55 10.10.20.20 66
Nov 25 11:06:00 10.10.20.20 66
Nov 25 11:06:05 10.10.20.20 2e
Nov 25 11:06:10 10.10.20.20 0f
Nov 25 11:06:15 10.10.20.20 1f
Nov 25 11:06:20 10.10.20.20 84
Nov 25 11:06:25 10.10.20.20 00
Nov 25 11:06:30 10.10.20.20 00
Nov 25 11:06:35 10.10.20.20 00
Nov 25 11:06:40 10.10.20.20 00
Nov 25 11:06:45 10.10.20.20 00
Nov 25 11:06:50 10.10.20.20 0f
Nov 25 11:06:55 10.10.20.20 b7
Nov 25 11:07:00 10.10.20.20 07
Nov 25 11:07:05 10.10.20.20 38
Nov 25 11:07:10 10.10.20.20 e0
Nov 25 11:07:15 10.10.20.20
Nov 25 11:07:21 10.10.20.20 [ 1236.099000] BUG: soft lockup - CPU#1
stuck for 67s! [pvscan:6172]
Nov 25 11:07:26 10.10.20.20 [ 1236.099001] Stack:
Nov 25 11:07:31 10.10.20.20 [ 1236.099001] Call Trace:
Nov 25 11:07:36 10.10.20.20 [ 1236.099001] Code:
Nov 25 11:07:41 10.10.20.20 39
Nov 25 11:07:46 10.10.20.20 ea
Nov 25 11:07:51 10.10.20.20 0f
Nov 25 11:07:56 10.10.20.20 1f
Nov 25 11:08:01 10.10.20.20 00
Nov 25 11:08:06 10.10.20.20 77
Nov 25 11:08:11 10.10.20.20 a8
Nov 25 11:08:16 10.10.20.20 89
Nov 25 11:08:21 10.10.20.20 de
Nov 25 11:08:26 10.10.20.20 89
Nov 25 11:08:31 10.10.20.20 e9
Nov 25 11:08:36 10.10.20.20 49
Nov 25 11:08:41 10.10.20.20 8d
Nov 25 11:08:46 10.10.20.20 34
Nov 25 11:08:51 10.10.20.20 f7
Nov 25 11:08:56 10.10.20.20 29
Nov 25 11:09:01 10.10.20.20 d9
Nov 25 11:09:06 10.10.20.20 45
Nov 25 11:09:11 10.10.20.20 89
Nov 25 11:09:16 10.10.20.20 e1
Nov 25 11:09:21 10.10.20.20 4c
Nov 25 11:09:26 10.10.20.20 8d
Nov 25 11:09:31 10.10.20.20 44
Nov 25 11:09:36 10.10.20.20 24
Nov 25 11:09:41 10.10.20.20 08
the sysrq-T just print "showstate" on screen. no more useful
information.
thanks.
Johnson
在 2010-11-25四的 07:01 +1100，Neil Brown写道：
> On Wed, 24 Nov 2010 22:34:21 +0800 Du Jun <dujun@perabytes.com> wrote:
>
> > Their's no dmesg information at all before the hang. The system just
> > hangs suddenly without any kernel printk or oops.
> >
> > mdadm version is debian lenny version 2.6.7.2-3
> >
> > I will get mdadm -E & mdadm -D informations tomorrow but I doubt they
> > are useful.
>
> Probably not, but they wouldn't hurt either.
>
> What would really help is if you could capture the result of
> sysrq-T
>
> which I accept might be a challenge depending on how hard the machine really
> has frozen.
>
> Also, make sure the LOCKUP_DETECTOR config options are set, and wait a few
> minutes after the hand to see if something pops up.
>
> I'll try to see if I can reproduce, but I don't have 16 disks so it may not
> work.
>
> NeilBrown
>
>
> >
> > Johnson
> > on 2010/11/24 22:19, Mathias Burén wrote:
> > > Hi,
> > >
> > > I suppose a few logs would be interesting. (dmesg, mdadm -E (hdds),
> > > mdadm -D md10, zcat /proc/config.gz, mdadm version)
> > >
> > > Regards,
> > > // Mathias
> > >
> > > 2010/11/24 Du Jun<dujun@perabytes.com>:
> > >
> > >> Hi,
> > >> We just quick tested the raid stability under the vanilla 2.6.36 kernel
> > >> and got an unstable result.
> > >>
> > >> first, we create a raid5 array using 16 sata disks:
> > >>
> > >> mdadm -C /dev/md10 -l 5 -n 16 /dev/sd[b-q]
> > >>
> > >> then use dd to stress the io:
> > >>
> > >> dd if=/dev/zero of=/dev/md10 bs=1M
> > >>
> > >> After a while, usually several minutes, the system hangs. It looks like
> > >> some kind of kernel deadlock. ping to this machine could get timely
> > >> response, however, any other process just hangs.
> > >>
> > >> It is easily reproducible and everytime the test result is a system hang.
> > >>
> > >> Johnson
> > >>
> > >> --
> > >> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > >> the body of a message to majordomo@vger.kernel.org
> > >> More majordomo info at http://vger.kernel.org/majordomo-info.html
> > >>
> > >>
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > >
> > >
> > >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
[-- Attachment #2: md10.txt --]
[-- Type: text/plain, Size: 1739 bytes --]
/dev/md10:
Version : 00.90
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Persistence : Superblock is persistent
Update Time : Thu Nov 25 10:42:10 2010
State : clean, degraded, recovering
Active Devices : 15
Working Devices : 16
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 64K
Rebuild Status : 2% complete
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Events : 0.4
Number Major Minor RaidDevice State
0 8 16 0 active sync /dev/sdb
1 8 32 1 active sync /dev/sdc
2 8 48 2 active sync /dev/sdd
3 8 64 3 active sync /dev/sde
4 8 80 4 active sync /dev/sdf
5 8 96 5 active sync /dev/sdg
6 8 112 6 active sync /dev/sdh
7 8 128 7 active sync /dev/sdi
8 8 144 8 active sync /dev/sdj
9 8 160 9 active sync /dev/sdk
10 8 176 10 active sync /dev/sdl
11 8 192 11 active sync /dev/sdm
12 8 208 12 active sync /dev/sdn
13 8 224 13 active sync /dev/sdo
14 8 240 14 active sync /dev/sdp
16 65 0 15 spare rebuilding /dev/sdq
[-- Attachment #3: md11.txt --]
[-- Type: text/plain, Size: 1746 bytes --]
/dev/md11:
Version : 00.90
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Persistence : Superblock is persistent
Update Time : Thu Nov 25 10:43:51 2010
State : clean, degraded, recovering
Active Devices : 15
Working Devices : 16
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 64K
Rebuild Status : 2% complete
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Events : 0.2
Number Major Minor RaidDevice State
0 65 16 0 active sync /dev/sdr
1 65 32 1 active sync /dev/sds
2 65 48 2 active sync /dev/sdt
3 65 64 3 active sync /dev/sdu
4 65 80 4 active sync /dev/sdv
5 65 96 5 active sync /dev/sdw
6 65 112 6 active sync /dev/sdx
7 65 128 7 active sync /dev/sdy
8 65 144 8 active sync /dev/sdz
9 65 160 9 active sync /dev/sdaa
10 65 176 10 active sync /dev/sdab
11 65 192 11 active sync /dev/sdac
12 65 208 12 active sync /dev/sdad
13 65 224 13 active sync /dev/sdae
14 65 240 14 active sync /dev/sdaf
16 66 0 15 spare rebuilding /dev/sdag
[-- Attachment #4: sdds.txt --]
[-- Type: text/plain, Size: 59378 bytes --]
/dev/sdaa:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d61 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 9 65 160 9 active sync /dev/sdaa
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdab:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d73 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 10 65 176 10 active sync /dev/sdab
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdac:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d85 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 11 65 192 11 active sync /dev/sdac
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdad:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d97 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 12 65 208 12 active sync /dev/sdad
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdae:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62da9 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 13 65 224 13 active sync /dev/sdae
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdaf:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62dbb - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 14 65 240 14 active sync /dev/sdaf
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdag:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62cca - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 16 66 0 16 spare /dev/sdag
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdb:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a4090 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 0 8 16 0 active sync /dev/sdb
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdc:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a40a2 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 1 8 32 1 active sync /dev/sdc
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdd:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a40b4 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 2 8 48 2 active sync /dev/sdd
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sde:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a40c6 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 3 8 64 3 active sync /dev/sde
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdf:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a40d8 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 4 8 80 4 active sync /dev/sdf
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdg:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a40ea - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 5 8 96 5 active sync /dev/sdg
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdh:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a40fc - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 6 8 112 6 active sync /dev/sdh
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdi:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a410e - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 7 8 128 7 active sync /dev/sdi
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdj:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a4120 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 8 8 144 8 active sync /dev/sdj
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdk:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a4132 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 9 8 160 9 active sync /dev/sdk
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdl:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a4144 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 10 8 176 10 active sync /dev/sdl
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdm:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a4156 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 11 8 192 11 active sync /dev/sdm
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdn:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a4168 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 12 8 208 12 active sync /dev/sdn
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdo:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a417a - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 13 8 224 13 active sync /dev/sdo
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdp:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a418c - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 14 8 240 14 active sync /dev/sdp
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdq:
Magic : a92b4efc
Version : 00.90.00
UUID : f7187a5e:3e368fe2:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:24:11 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 10
Update Time : Thu Nov 25 10:51:49 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : aa5a40d3 - correct
Events : 6
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 16 65 0 16 spare /dev/sdq
0 0 8 16 0 active sync /dev/sdb
1 1 8 32 1 active sync /dev/sdc
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 96 5 active sync /dev/sdg
6 6 8 112 6 active sync /dev/sdh
7 7 8 128 7 active sync /dev/sdi
8 8 8 144 8 active sync /dev/sdj
9 9 8 160 9 active sync /dev/sdk
10 10 8 176 10 active sync /dev/sdl
11 11 8 192 11 active sync /dev/sdm
12 12 8 208 12 active sync /dev/sdn
13 13 8 224 13 active sync /dev/sdo
14 14 8 240 14 active sync /dev/sdp
15 15 0 0 15 faulty removed
16 16 65 0 16 spare /dev/sdq
/dev/sdr:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62cbf - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 0 65 16 0 active sync /dev/sdr
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sds:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62cd1 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 1 65 32 1 active sync /dev/sds
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdt:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62ce3 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 2 65 48 2 active sync /dev/sdt
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdu:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62cf5 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 3 65 64 3 active sync /dev/sdu
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdv:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d07 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 4 65 80 4 active sync /dev/sdv
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdw:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d19 - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 5 65 96 5 active sync /dev/sdw
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdx:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d2b - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 6 65 112 6 active sync /dev/sdx
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdy:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d3d - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 7 65 128 7 active sync /dev/sdy
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
/dev/sdz:
Magic : a92b4efc
Version : 00.90.00
UUID : 012dcc5b:f09d21e4:d64d9ae0:217d735e (local to host DigiOcean)
Creation Time : Thu Nov 25 10:43:51 2010
Raid Level : raid5
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Array Size : 14651437440 (13972.70 GiB 15003.07 GB)
Raid Devices : 16
Total Devices : 16
Preferred Minor : 11
Update Time : Thu Nov 25 10:51:55 2010
State : clean
Active Devices : 15
Working Devices : 16
Failed Devices : 1
Spare Devices : 1
Checksum : 66d62d4f - correct
Events : 4
Layout : left-symmetric
Chunk Size : 64K
Number Major Minor RaidDevice State
this 8 65 144 8 active sync /dev/sdz
0 0 65 16 0 active sync /dev/sdr
1 1 65 32 1 active sync /dev/sds
2 2 65 48 2 active sync /dev/sdt
3 3 65 64 3 active sync /dev/sdu
4 4 65 80 4 active sync /dev/sdv
5 5 65 96 5 active sync /dev/sdw
6 6 65 112 6 active sync /dev/sdx
7 7 65 128 7 active sync /dev/sdy
8 8 65 144 8 active sync /dev/sdz
9 9 65 160 9 active sync /dev/sdaa
10 10 65 176 10 active sync /dev/sdab
11 11 65 192 11 active sync /dev/sdac
12 12 65 208 12 active sync /dev/sdad
13 13 65 224 13 active sync /dev/sdae
14 14 65 240 14 active sync /dev/sdaf
15 15 0 0 15 faulty removed
16 16 66 0 16 spare /dev/sdag
next prev parent reply other threads:[~2010-11-25 3:12 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-24 13:48 raid related kernel hang in 2.6.36 Du Jun
2010-11-24 14:19 ` Mathias Burén
2010-11-24 14:34 ` Du Jun
2010-11-24 20:01 ` Neil Brown
2010-11-24 21:15 ` Carlos Carvalho
2010-11-25 3:12 ` dujun [this message]
2011-02-10 16:28 ` Iordan Iordanov
2011-02-11 15:52 ` Iordan Iordanov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1290654763.5416.31.camel@dujun.perabytes.com \
--to=dujun@perabytes.com \
--cc=linux-raid@vger.kernel.org \
--cc=mathias.buren@gmail.com \
--cc=neilb@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.