RAID6 array won't start

Linux RAID subsystem development
 help / color / mirror / Atom feed

* RAID6 array won't start
       [not found] <AANLkTimLfJHc2waPkvjUXn=mewc37qkCd-r0GG4k=e-D@mail.gmail.com>
@ 2011-02-11 16:48 ` Christian
  2011-02-11 20:49   ` NeilBrown
  0 siblings, 1 reply; 11+ messages in thread
From: Christian @ 2011-02-11 16:48 UTC (permalink / raw)
  To: linux-raid

Hello,

I have a 10 disk RAID6 array.  I had an issue with a molex connector
that caused drives to drop from a 5in3 SATA backplane.  There wasn't
any activity occurring on the array during the time, so I think the
filesystem should be in tact.  The issue now is the event counts are
out of sync, along with the state of each drive.

# mdadm -A /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
/dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
mdadm: /dev/md1 assembled from 8 drives - not enough to start the
array while not clean - consider --force.

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md1 : inactive sdt1[1](S) sde1[9](S) sdl1[8](S) sdg1[7](S) sdf1[6](S)
sdk1[5](S) sdq1[4](S) sds1[3](S) sdr1[2](S) sdm1[0](S)
      19535134720 blocks


# mdadm -Af /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
/dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
mdadm: failed to RUN_ARRAY /dev/md1: Input/output error

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md1 : inactive sdt1[1] sde1[9] sdl1[8] sdg1[7] sdf1[6] sdk1[5] sdq1[4] sdr1[2]
      15628107776 blocks

I've read others having success with this but it's not accepting it.
# echo "clean" > /sys/block/md1/md/array_state
bash: echo: write error: Invalid argument


# mdadm -E /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1 /dev/sdm1
/dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1 | egrep Event
         Events : 147226
         Events : 147226
         Events : 147226
         Events : 147226
         Events : 147226
         Events : 147221
         Events : 147225
         Events : 147225
         Events : 147221
         Events : 147225


# mdadm -E /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1 /dev/sdm1
/dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27a7ead3 - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     9       8       65        9      active sync   /dev/sde1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdf1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27a7eadd - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     6       8       81        6      active sync   /dev/sdf1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdg1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27a7eaef - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     7       8       97        7      active sync   /dev/sdg1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdk1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27a7eb2b - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     5       8      161        5      active sync   /dev/sdk1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdl1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27a7eb41 - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     8       8      177        8      active sync   /dev/sdl1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdm1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 10
Preferred Minor : 1

    Update Time : Wed Feb  9 18:45:59 2011
          State : clean
 Active Devices : 10
Working Devices : 10
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27aa228c - correct
         Events : 147221

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     0       8      193        0      active sync   /dev/sdm1

   0     0       8      193        0      active sync   /dev/sdm1
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3      65       33        3      active sync   /dev/sds1
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdq1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:20:33 2011
          State : clean
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27aa2903 - correct
         Events : 147225

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     4      65        1        4      active sync   /dev/sdq1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdr1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:20:33 2011
          State : clean
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27aa290f - correct
         Events : 147225

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     2      65       17        2      active sync   /dev/sdr1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sds1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 10
Preferred Minor : 1

    Update Time : Wed Feb  9 18:45:59 2011
          State : clean
 Active Devices : 10
Working Devices : 10
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27aa222b - correct
         Events : 147221

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     3      65       33        3      active sync   /dev/sds1

   0     0       8      193        0      active sync   /dev/sdm1
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3      65       33        3      active sync   /dev/sds1
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdt1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:20:33 2011
          State : clean
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27aa292d - correct
         Events : 147225

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     1      65       49        1      active sync   /dev/sdt1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1


I've read others who have had success or failure using create to get
things back in order again.  This is what I came up with the drives
listed in sequence from 0 to 9.  The drives marked missing are the two
that have event counts much lower than the rest.

# mdadm --create /dev/md1 --verbose --level=6 --raid-devices=10
--chunk=64 missing /dev/sdt1 /dev/sdr1 missing /dev/sdq1 /dev/sdk1
/dev/sdf1 /dev/sdg1 /dev/sdl1 /dev/sde1
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: /dev/sdt1 appears to contain an ext2fs file system
    size=-1551761408K  mtime=Wed Feb  9 18:45:59 2011
mdadm: /dev/sdt1 appears to be part of a raid array:
    level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
mdadm: layout defaults to left-symmetric
mdadm: /dev/sdr1 appears to be part of a raid array:
    level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
mdadm: layout defaults to left-symmetric
mdadm: /dev/sdq1 appears to be part of a raid array:
    level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
mdadm: layout defaults to left-symmetric
mdadm: /dev/sdk1 appears to be part of a raid array:
    level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
mdadm: layout defaults to left-symmetric
mdadm: /dev/sdf1 appears to be part of a raid array:
    level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
mdadm: layout defaults to left-symmetric
mdadm: /dev/sdg1 appears to be part of a raid array:
    level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
mdadm: layout defaults to left-symmetric
mdadm: /dev/sdl1 appears to be part of a raid array:
    level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
mdadm: layout defaults to left-symmetric
mdadm: /dev/sde1 appears to contain an ext2fs file system
    size=-591265728K  mtime=Mon Feb 14 13:17:27 2028
mdadm: /dev/sde1 appears to be part of a raid array:
    level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
mdadm: size set to 1953512384K
Continue creating array? n

I haven't executed this yet though.


Any help would be appreciated,
Thanks

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-11 16:48 ` RAID6 array won't start Christian
@ 2011-02-11 20:49   ` NeilBrown
  2011-02-11 21:13     ` Christian
  0 siblings, 1 reply; 11+ messages in thread
From: NeilBrown @ 2011-02-11 20:49 UTC (permalink / raw)
  To: Christian; +Cc: linux-raid

On Fri, 11 Feb 2011 11:48:41 -0500 Christian <cjs703@gmail.com> wrote:

> Hello,
> 
> I have a 10 disk RAID6 array.  I had an issue with a molex connector
> that caused drives to drop from a 5in3 SATA backplane.  There wasn't
> any activity occurring on the array during the time, so I think the
> filesystem should be in tact.  The issue now is the event counts are
> out of sync, along with the state of each drive.
> 
> # mdadm -A /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> mdadm: /dev/md1 assembled from 8 drives - not enough to start the
> array while not clean - consider --force.
> 
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md1 : inactive sdt1[1](S) sde1[9](S) sdl1[8](S) sdg1[7](S) sdf1[6](S)
> sdk1[5](S) sdq1[4](S) sds1[3](S) sdr1[2](S) sdm1[0](S)
>       19535134720 blocks
> 
> 
> # mdadm -Af /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> mdadm: failed to RUN_ARRAY /dev/md1: Input/output error

The array is already in a partially-assembled state.

mdadm -S /dev/md1

before trying to assemble again with -f.

NeilBrown


> 
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md1 : inactive sdt1[1] sde1[9] sdl1[8] sdg1[7] sdf1[6] sdk1[5] sdq1[4] sdr1[2]
>       15628107776 blocks
> 
> I've read others having success with this but it's not accepting it.
> # echo "clean" > /sys/block/md1/md/array_state
> bash: echo: write error: Invalid argument
> 
> 
> # mdadm -E /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1 /dev/sdm1
> /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1 | egrep Event
>          Events : 147226
>          Events : 147226
>          Events : 147226
>          Events : 147226
>          Events : 147226
>          Events : 147221
>          Events : 147225
>          Events : 147225
>          Events : 147221
>          Events : 147225
> 
> 
> # mdadm -E /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1 /dev/sdm1
> /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 8
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 19:24:09 2011
>           State : active
>  Active Devices : 8
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27a7ead3 - correct
>          Events : 147226
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     9       8       65        9      active sync   /dev/sde1
> 
>    0     0       0        0        0      removed
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3       0        0        3      faulty removed
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sdf1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 8
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 19:24:09 2011
>           State : active
>  Active Devices : 8
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27a7eadd - correct
>          Events : 147226
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     6       8       81        6      active sync   /dev/sdf1
> 
>    0     0       0        0        0      removed
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3       0        0        3      faulty removed
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sdg1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 8
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 19:24:09 2011
>           State : active
>  Active Devices : 8
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27a7eaef - correct
>          Events : 147226
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     7       8       97        7      active sync   /dev/sdg1
> 
>    0     0       0        0        0      removed
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3       0        0        3      faulty removed
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sdk1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 8
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 19:24:09 2011
>           State : active
>  Active Devices : 8
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27a7eb2b - correct
>          Events : 147226
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     5       8      161        5      active sync   /dev/sdk1
> 
>    0     0       0        0        0      removed
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3       0        0        3      faulty removed
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sdl1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 8
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 19:24:09 2011
>           State : active
>  Active Devices : 8
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27a7eb41 - correct
>          Events : 147226
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     8       8      177        8      active sync   /dev/sdl1
> 
>    0     0       0        0        0      removed
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3       0        0        3      faulty removed
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sdm1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 10
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 18:45:59 2011
>           State : clean
>  Active Devices : 10
> Working Devices : 10
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27aa228c - correct
>          Events : 147221
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     0       8      193        0      active sync   /dev/sdm1
> 
>    0     0       8      193        0      active sync   /dev/sdm1
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3      65       33        3      active sync   /dev/sds1
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sdq1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 8
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 19:20:33 2011
>           State : clean
>  Active Devices : 8
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27aa2903 - correct
>          Events : 147225
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     4      65        1        4      active sync   /dev/sdq1
> 
>    0     0       0        0        0      removed
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3       0        0        3      faulty removed
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sdr1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 8
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 19:20:33 2011
>           State : clean
>  Active Devices : 8
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27aa290f - correct
>          Events : 147225
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     2      65       17        2      active sync   /dev/sdr1
> 
>    0     0       0        0        0      removed
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3       0        0        3      faulty removed
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sds1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 10
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 18:45:59 2011
>           State : clean
>  Active Devices : 10
> Working Devices : 10
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27aa222b - correct
>          Events : 147221
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     3      65       33        3      active sync   /dev/sds1
> 
>    0     0       8      193        0      active sync   /dev/sdm1
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3      65       33        3      active sync   /dev/sds1
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> /dev/sdt1:
>           Magic : a92b4efc
>         Version : 0.90.00
>            UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
>   Creation Time : Sat Sep 25 13:23:40 2010
>      Raid Level : raid6
>   Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
>      Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
>    Raid Devices : 10
>   Total Devices : 8
> Preferred Minor : 1
> 
>     Update Time : Wed Feb  9 19:20:33 2011
>           State : clean
>  Active Devices : 8
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 0
>        Checksum : 27aa292d - correct
>          Events : 147225
> 
>          Layout : left-symmetric
>      Chunk Size : 64K
> 
>       Number   Major   Minor   RaidDevice State
> this     1      65       49        1      active sync   /dev/sdt1
> 
>    0     0       0        0        0      removed
>    1     1      65       49        1      active sync   /dev/sdt1
>    2     2      65       17        2      active sync   /dev/sdr1
>    3     3       0        0        3      faulty removed
>    4     4      65        1        4      active sync   /dev/sdq1
>    5     5       8      161        5      active sync   /dev/sdk1
>    6     6       8       81        6      active sync   /dev/sdf1
>    7     7       8       97        7      active sync   /dev/sdg1
>    8     8       8      177        8      active sync   /dev/sdl1
>    9     9       8       65        9      active sync   /dev/sde1
> 
> 
> I've read others who have had success or failure using create to get
> things back in order again.  This is what I came up with the drives
> listed in sequence from 0 to 9.  The drives marked missing are the two
> that have event counts much lower than the rest.
> 
> # mdadm --create /dev/md1 --verbose --level=6 --raid-devices=10
> --chunk=64 missing /dev/sdt1 /dev/sdr1 missing /dev/sdq1 /dev/sdk1
> /dev/sdf1 /dev/sdg1 /dev/sdl1 /dev/sde1
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdt1 appears to contain an ext2fs file system
>     size=-1551761408K  mtime=Wed Feb  9 18:45:59 2011
> mdadm: /dev/sdt1 appears to be part of a raid array:
>     level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdr1 appears to be part of a raid array:
>     level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdq1 appears to be part of a raid array:
>     level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdk1 appears to be part of a raid array:
>     level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdf1 appears to be part of a raid array:
>     level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdg1 appears to be part of a raid array:
>     level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sdl1 appears to be part of a raid array:
>     level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sde1 appears to contain an ext2fs file system
>     size=-591265728K  mtime=Mon Feb 14 13:17:27 2028
> mdadm: /dev/sde1 appears to be part of a raid array:
>     level=raid6 devices=10 ctime=Sat Sep 25 13:23:40 2010
> mdadm: size set to 1953512384K
> Continue creating array? n
> 
> I haven't executed this yet though.
> 
> 
> Any help would be appreciated,
> Thanks
> N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±¨_Š{ayº\x1dÊ‡Ú™ë,j\a¢f£¢·hš‹àz¹\x1e®w¥¢¸\f¢·¦j:+v‰¨ŠwèjØm¶Ÿÿ¾\a«‘êçzZ+ƒùšŽŠÝ¢j"_ú!
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-11 20:49   ` NeilBrown
@ 2011-02-11 21:13     ` Christian
  2011-02-11 21:32       ` NeilBrown
  0 siblings, 1 reply; 11+ messages in thread
From: Christian @ 2011-02-11 21:13 UTC (permalink / raw)
  To: NeilBrown; +Cc: linux-raid

On Fri, Feb 11, 2011 at 3:49 PM, NeilBrown <neilb@suse.de> wrote:
> On Fri, 11 Feb 2011 11:48:41 -0500 Christian <cjs703@gmail.com> wrote:
>
>> Hello,
>>
>> I have a 10 disk RAID6 array.  I had an issue with a molex connector
>> that caused drives to drop from a 5in3 SATA backplane.  There wasn't
>> any activity occurring on the array during the time, so I think the
>> filesystem should be in tact.  The issue now is the event counts are
>> out of sync, along with the state of each drive.
>>
>> # mdadm -A /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
>> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
>> mdadm: /dev/md1 assembled from 8 drives - not enough to start the
>> array while not clean - consider --force.
>>
>> # cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4]
>> md1 : inactive sdt1[1](S) sde1[9](S) sdl1[8](S) sdg1[7](S) sdf1[6](S)
>> sdk1[5](S) sdq1[4](S) sds1[3](S) sdr1[2](S) sdm1[0](S)
>>       19535134720 blocks
>>
>>
>> # mdadm -Af /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
>> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
>> mdadm: failed to RUN_ARRAY /dev/md1: Input/output error
>
> The array is already in a partially-assembled state.
>
> mdadm -S /dev/md1
>
> before trying to assemble again with -f.
>
> NeilBrown

I should have probably included it but I did stop the array with -S
before running -Af.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-11 21:13     ` Christian
@ 2011-02-11 21:32       ` NeilBrown
  2011-02-11 21:51         ` Christian
  0 siblings, 1 reply; 11+ messages in thread
From: NeilBrown @ 2011-02-11 21:32 UTC (permalink / raw)
  To: Christian; +Cc: linux-raid

On Fri, 11 Feb 2011 16:13:53 -0500 Christian <cjs703@gmail.com> wrote:

> On Fri, Feb 11, 2011 at 3:49 PM, NeilBrown <neilb@suse.de> wrote:
> > On Fri, 11 Feb 2011 11:48:41 -0500 Christian <cjs703@gmail.com> wrote:
> >
> >> Hello,
> >>
> >> I have a 10 disk RAID6 array.  I had an issue with a molex connector
> >> that caused drives to drop from a 5in3 SATA backplane.  There wasn't
> >> any activity occurring on the array during the time, so I think the
> >> filesystem should be in tact.  The issue now is the event counts are
> >> out of sync, along with the state of each drive.
> >>
> >> # mdadm -A /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
> >> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> >> mdadm: /dev/md1 assembled from 8 drives - not enough to start the
> >> array while not clean - consider --force.
> >>
> >> # cat /proc/mdstat
> >> Personalities : [raid6] [raid5] [raid4]
> >> md1 : inactive sdt1[1](S) sde1[9](S) sdl1[8](S) sdg1[7](S) sdf1[6](S)
> >> sdk1[5](S) sdq1[4](S) sds1[3](S) sdr1[2](S) sdm1[0](S)
> >>       19535134720 blocks
> >>
> >>
> >> # mdadm -Af /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1 /dev/sdl1
> >> /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1
> >> mdadm: failed to RUN_ARRAY /dev/md1: Input/output error
> >
> > The array is already in a partially-assembled state.
> >
> > mdadm -S /dev/md1
> >
> > before trying to assemble again with -f.
> >
> > NeilBrown
> 
> I should have probably included it but I did stop the array with -S
> before running -Af.

True.

Ok, being sure that the array is stopped first, try the "-Af" command again,
but an a -v.
Then post the results together with any kernel messages (dmesg | tail -50
maybe)

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-11 21:32       ` NeilBrown
@ 2011-02-11 21:51         ` Christian
  2011-02-11 22:36           ` Christian
  0 siblings, 1 reply; 11+ messages in thread
From: Christian @ 2011-02-11 21:51 UTC (permalink / raw)
  To: NeilBrown; +Cc: linux-raid

> True.
>
> Ok, being sure that the array is stopped first, try the "-Af" command again,
> but an a -v.
> Then post the results together with any kernel messages (dmesg | tail -50
> maybe)
>
> NeilBrown
>

 # mdadm -Af -v /dev/md1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdk1
/dev/sdl1 /dev/sdm1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1mdadm:
looking for devices for /dev/md1
mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 9.
mdadm: /dev/sdf1 is identified as a member of /dev/md1, slot 6.
mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 7.
mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 5.
mdadm: /dev/sdl1 is identified as a member of /dev/md1, slot 8.
mdadm: /dev/sdm1 is identified as a member of /dev/md1, slot 0.
mdadm: /dev/sdq1 is identified as a member of /dev/md1, slot 4.
mdadm: /dev/sdr1 is identified as a member of /dev/md1, slot 2.
mdadm: /dev/sds1 is identified as a member of /dev/md1, slot 3.
mdadm: /dev/sdt1 is identified as a member of /dev/md1, slot 1.
mdadm: added /dev/sdm1 to /dev/md1 as 0
mdadm: added /dev/sdr1 to /dev/md1 as 2
mdadm: added /dev/sds1 to /dev/md1 as 3
mdadm: added /dev/sdq1 to /dev/md1 as 4
mdadm: added /dev/sdk1 to /dev/md1 as 5
mdadm: added /dev/sdf1 to /dev/md1 as 6
mdadm: added /dev/sdg1 to /dev/md1 as 7
mdadm: added /dev/sdl1 to /dev/md1 as 8
mdadm: added /dev/sde1 to /dev/md1 as 9
mdadm: added /dev/sdt1 to /dev/md1 as 1
mdadm: failed to RUN_ARRAY /dev/md1: Input/output error

[68107.441660] md/raid:md1: not clean -- starting background reconstruction
[68107.441669] md/raid:md1: device sdt1 operational as raid disk 1
[68107.441671] md/raid:md1: device sde1 operational as raid disk 9
[68107.441673] md/raid:md1: device sdl1 operational as raid disk 8
[68107.441674] md/raid:md1: device sdg1 operational as raid disk 7
[68107.441676] md/raid:md1: device sdf1 operational as raid disk 6
[68107.441678] md/raid:md1: device sdk1 operational as raid disk 5
[68107.441679] md/raid:md1: device sdq1 operational as raid disk 4
[68107.441681] md/raid:md1: device sdr1 operational as raid disk 2
[68107.442368] md/raid:md1: allocated 10572kB
[68107.442388] md/raid:md1: cannot start dirty degraded array.
[68107.442394] RAID conf printout:
[68107.442396]  --- level:6 rd:10 wd:8
[68107.442397]  disk 1, o:1, dev:sdt1
[68107.442399]  disk 2, o:1, dev:sdr1
[68107.442400]  disk 4, o:1, dev:sdq1
[68107.442402]  disk 5, o:1, dev:sdk1
[68107.442403]  disk 6, o:1, dev:sdf1
[68107.442404]  disk 7, o:1, dev:sdg1
[68107.442406]  disk 8, o:1, dev:sdl1
[68107.442407]  disk 9, o:1, dev:sde1
[68107.442692] md/raid:md1: failed to run raid set.
[68107.442693] md: pers->run() failed ...

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-11 21:51         ` Christian
@ 2011-02-11 22:36           ` Christian
  2011-02-11 23:37             ` Christian
  2011-02-12  0:52             ` NeilBrown
  0 siblings, 2 replies; 11+ messages in thread
From: Christian @ 2011-02-11 22:36 UTC (permalink / raw)
  To: NeilBrown; +Cc: linux-raid

Here it is again, there were a few other messages in there at the beginning.

[70745.855861] md: md1 stopped.
[70745.859303] md: bind<sdm1>
[70745.859576] md: bind<sdr1>
[70745.859720] md: bind<sds1>
[70745.860265] md: bind<sdq1>
[70745.860428] md: bind<sdk1>
[70745.860680] md: bind<sdf1>
[70745.860839] md: bind<sdg1>
[70745.860986] md: bind<sdl1>
[70745.861169] md: bind<sde1>
[70745.861359] md: bind<sdt1>
[70745.861407] md: kicking non-fresh sds1 from array!
[70745.861411] md: unbind<sds1>
[70745.870570] md: export_rdev(sds1)
[70745.870598] md: kicking non-fresh sdm1 from array!
[70745.870602] md: unbind<sdm1>
[70745.882561] md: export_rdev(sdm1)
[70745.883324] md/raid:md1: not clean -- starting background reconstruction
[70745.883333] md/raid:md1: device sdt1 operational as raid disk 1
[70745.883335] md/raid:md1: device sde1 operational as raid disk 9
[70745.883337] md/raid:md1: device sdl1 operational as raid disk 8
[70745.883339] md/raid:md1: device sdg1 operational as raid disk 7
[70745.883340] md/raid:md1: device sdf1 operational as raid disk 6
[70745.883342] md/raid:md1: device sdk1 operational as raid disk 5
[70745.883344] md/raid:md1: device sdq1 operational as raid disk 4
[70745.883345] md/raid:md1: device sdr1 operational as raid disk 2
[70745.884038] md/raid:md1: allocated 10572kB
[70745.884057] md/raid:md1: cannot start dirty degraded array.
[70745.884065] RAID conf printout:
[70745.884066]  --- level:6 rd:10 wd:8
[70745.884068]  disk 1, o:1, dev:sdt1
[70745.884069]  disk 2, o:1, dev:sdr1
[70745.884070]  disk 4, o:1, dev:sdq1
[70745.884072]  disk 5, o:1, dev:sdk1
[70745.884073]  disk 6, o:1, dev:sdf1
[70745.884074]  disk 7, o:1, dev:sdg1
[70745.884076]  disk 8, o:1, dev:sdl1
[70745.884077]  disk 9, o:1, dev:sde1
[70745.884370] md/raid:md1: failed to run raid set.
[70745.884372] md: pers->run() failed ...

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-11 22:36           ` Christian
@ 2011-02-11 23:37             ` Christian
  2011-02-12  0:52             ` NeilBrown
  1 sibling, 0 replies; 11+ messages in thread
From: Christian @ 2011-02-11 23:37 UTC (permalink / raw)
  To: NeilBrown; +Cc: linux-raid

Ok problem solved!! The array and data are back!! With the help of
some of the guys on #linux-raid on freenode I went ahead and ran
--create on the array.  If anyone else happens to have the same issue
here is what I did.

First I backed up all the examine info by doing:
# mdadm -E /dev/sd[efgklmqrst]1 > examineinfo

And then I backed up the superblocks at the end of the disk by first
running fdisk to get the size, and then using dd
# fdisk /dev/sdt
   Device Boot      Start         End      Blocks   Id  System
/dev/sdt1            2048  3907029167  1953513560   fd  Linux raid autodetect

Took the total block numbers and then just skipped to the end by
dropping a 0 in the thousands column.  It's not exactly the superblock
but should be enough to have a backup in case things get really nasty
later.

# for i in sdm sdt sdr sds sdq sdk sdf sdg sdl sde; do dd if=/dev/$i
of=$i.tail skip=1953510560 bs=1k; done

And then I issued the create command, making sure it matched the info
from examine, and the drives letters were in order sequentially from 0
to 9, leaving out the two disks that dropped originally that were
marked with lower event counts.

# mdadm --create --metadata=0.90 --assume-clean --verbose --level=6
--raid-devices=10 --chunk=64 /dev/md1 missing /dev/sdt1 /dev/sdr1
missing /dev/sdq1 /dev/sdk1 /dev/sdf1 /dev/sdg1 /dev/sdl1 /dev/sde1

After executing this I have:
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md1 : active raid6 sde1[9] sdl1[8] sdg1[7] sdf1[6] sdk1[5] sdq1[4]
sdr1[2] sdt1[1]
      15628107776 blocks level 6, 64k chunk, algorithm 2 [10/8] [_UU_UUUUUU]

When attempting to mount the ext4 fs it initially failed, didn't
recognize the fs at all, but after:

# fsck /dev/md1

It fixed up the journal, etc and I'm now able to mount the file system
with all the data in tact!

I'm not completely out of the woods yet since I have to re-add 2
disks, but as of now everything looks great.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-11 22:36           ` Christian
  2011-02-11 23:37             ` Christian
@ 2011-02-12  0:52             ` NeilBrown
  2011-02-12  0:57               ` Christian
  1 sibling, 1 reply; 11+ messages in thread
From: NeilBrown @ 2011-02-12  0:52 UTC (permalink / raw)
  To: Christian; +Cc: linux-raid

On Fri, 11 Feb 2011 17:36:08 -0500 Christian <cjs703@gmail.com> wrote:

> Here it is again, there were a few other messages in there at the beginning.
> 
> [70745.855861] md: md1 stopped.
> [70745.859303] md: bind<sdm1>
> [70745.859576] md: bind<sdr1>
> [70745.859720] md: bind<sds1>
> [70745.860265] md: bind<sdq1>
> [70745.860428] md: bind<sdk1>
> [70745.860680] md: bind<sdf1>
> [70745.860839] md: bind<sdg1>
> [70745.860986] md: bind<sdl1>
> [70745.861169] md: bind<sde1>
> [70745.861359] md: bind<sdt1>
> [70745.861407] md: kicking non-fresh sds1 from array!
> [70745.861411] md: unbind<sds1>
> [70745.870570] md: export_rdev(sds1)
> [70745.870598] md: kicking non-fresh sdm1 from array!

That shouldn't happen when you use -f ....

Could be that you hit the mdadm bug fixed by commit  67a8c82d606
which is in mdadm-2.6
What version of mdadm are you running???

(I note you have got your data back by other means - good!)

NeilBrown



> [70745.870602] md: unbind<sdm1>
> [70745.882561] md: export_rdev(sdm1)
> [70745.883324] md/raid:md1: not clean -- starting background reconstruction
> [70745.883333] md/raid:md1: device sdt1 operational as raid disk 1
> [70745.883335] md/raid:md1: device sde1 operational as raid disk 9
> [70745.883337] md/raid:md1: device sdl1 operational as raid disk 8
> [70745.883339] md/raid:md1: device sdg1 operational as raid disk 7
> [70745.883340] md/raid:md1: device sdf1 operational as raid disk 6
> [70745.883342] md/raid:md1: device sdk1 operational as raid disk 5
> [70745.883344] md/raid:md1: device sdq1 operational as raid disk 4
> [70745.883345] md/raid:md1: device sdr1 operational as raid disk 2
> [70745.884038] md/raid:md1: allocated 10572kB
> [70745.884057] md/raid:md1: cannot start dirty degraded array.
> [70745.884065] RAID conf printout:
> [70745.884066]  --- level:6 rd:10 wd:8
> [70745.884068]  disk 1, o:1, dev:sdt1
> [70745.884069]  disk 2, o:1, dev:sdr1
> [70745.884070]  disk 4, o:1, dev:sdq1
> [70745.884072]  disk 5, o:1, dev:sdk1
> [70745.884073]  disk 6, o:1, dev:sdf1
> [70745.884074]  disk 7, o:1, dev:sdg1
> [70745.884076]  disk 8, o:1, dev:sdl1
> [70745.884077]  disk 9, o:1, dev:sde1
> [70745.884370] md/raid:md1: failed to run raid set.
> [70745.884372] md: pers->run() failed ...


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-12  0:52             ` NeilBrown
@ 2011-02-12  0:57               ` Christian
  2011-02-12  1:03                 ` NeilBrown
  0 siblings, 1 reply; 11+ messages in thread
From: Christian @ 2011-02-12  0:57 UTC (permalink / raw)
  To: NeilBrown; +Cc: linux-raid

# mdadm -V
mdadm - v3.1.4 - 31st August 2010

On Fri, Feb 11, 2011 at 7:52 PM, NeilBrown <neilb@suse.de> wrote:
> On Fri, 11 Feb 2011 17:36:08 -0500 Christian <cjs703@gmail.com> wrote:
>
>> Here it is again, there were a few other messages in there at the beginning.
>>
>> [70745.855861] md: md1 stopped.
>> [70745.859303] md: bind<sdm1>
>> [70745.859576] md: bind<sdr1>
>> [70745.859720] md: bind<sds1>
>> [70745.860265] md: bind<sdq1>
>> [70745.860428] md: bind<sdk1>
>> [70745.860680] md: bind<sdf1>
>> [70745.860839] md: bind<sdg1>
>> [70745.860986] md: bind<sdl1>
>> [70745.861169] md: bind<sde1>
>> [70745.861359] md: bind<sdt1>
>> [70745.861407] md: kicking non-fresh sds1 from array!
>> [70745.861411] md: unbind<sds1>
>> [70745.870570] md: export_rdev(sds1)
>> [70745.870598] md: kicking non-fresh sdm1 from array!
>
> That shouldn't happen when you use -f ....
>
> Could be that you hit the mdadm bug fixed by commit  67a8c82d606
> which is in mdadm-2.6
> What version of mdadm are you running???
>
> (I note you have got your data back by other means - good!)
>
> NeilBrown
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-12  0:57               ` Christian
@ 2011-02-12  1:03                 ` NeilBrown
  2011-02-12  4:08                   ` Christian
  0 siblings, 1 reply; 11+ messages in thread
From: NeilBrown @ 2011-02-12  1:03 UTC (permalink / raw)
  To: Christian; +Cc: linux-raid

On Fri, 11 Feb 2011 19:57:52 -0500 Christian <cjs703@gmail.com> wrote:

> # mdadm -V
> mdadm - v3.1.4 - 31st August 2010

So either I didn't fix it like I thought, or I subsequently broke it.

Could you send me the combined --examine output that you saved?
That might help me reproduce the exact problem, then add it to the
mdadm test suite.  Then fix it.

Thanks,
NeilBrown


> 
> On Fri, Feb 11, 2011 at 7:52 PM, NeilBrown <neilb@suse.de> wrote:
> > On Fri, 11 Feb 2011 17:36:08 -0500 Christian <cjs703@gmail.com> wrote:
> >
> >> Here it is again, there were a few other messages in there at the beginning.
> >>
> >> [70745.855861] md: md1 stopped.
> >> [70745.859303] md: bind<sdm1>
> >> [70745.859576] md: bind<sdr1>
> >> [70745.859720] md: bind<sds1>
> >> [70745.860265] md: bind<sdq1>
> >> [70745.860428] md: bind<sdk1>
> >> [70745.860680] md: bind<sdf1>
> >> [70745.860839] md: bind<sdg1>
> >> [70745.860986] md: bind<sdl1>
> >> [70745.861169] md: bind<sde1>
> >> [70745.861359] md: bind<sdt1>
> >> [70745.861407] md: kicking non-fresh sds1 from array!
> >> [70745.861411] md: unbind<sds1>
> >> [70745.870570] md: export_rdev(sds1)
> >> [70745.870598] md: kicking non-fresh sdm1 from array!
> >
> > That shouldn't happen when you use -f ....
> >
> > Could be that you hit the mdadm bug fixed by commit  67a8c82d606
> > which is in mdadm-2.6
> > What version of mdadm are you running???
> >
> > (I note you have got your data back by other means - good!)
> >
> > NeilBrown
> >

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RAID6 array won't start
  2011-02-12  1:03                 ` NeilBrown
@ 2011-02-12  4:08                   ` Christian
  0 siblings, 0 replies; 11+ messages in thread
From: Christian @ 2011-02-12  4:08 UTC (permalink / raw)
  To: NeilBrown; +Cc: linux-raid

Here's the combined output

/dev/sde1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 27a7ead4 - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     9       8       65        9      active sync   /dev/sde1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdf1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 27a7eade - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     6       8       81        6      active sync   /dev/sdf1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdg1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 27a7eaf0 - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     7       8       97        7      active sync   /dev/sdg1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdk1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 27a7eb2c - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     5       8      161        5      active sync   /dev/sdk1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdl1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:24:09 2011
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 27a7eb42 - correct
         Events : 147226

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     8       8      177        8      active sync   /dev/sdl1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdm1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 10
Preferred Minor : 1

    Update Time : Wed Feb  9 18:45:59 2011
          State : clean
 Active Devices : 10
Working Devices : 10
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27aa228c - correct
         Events : 147221

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     0       8      193        0      active sync   /dev/sdm1

   0     0       8      193        0      active sync   /dev/sdm1
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3      65       33        3      active sync   /dev/sds1
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdq1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:20:33 2011
          State : clean
 Active Devices : 8
Working Devices : 8
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 27aa2904 - correct
         Events : 147225

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     4      65        1        4      active sync   /dev/sdq1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdr1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:20:33 2011
          State : clean
 Active Devices : 8
Working Devices : 8
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 27aa2910 - correct
         Events : 147225

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     2      65       17        2      active sync   /dev/sdr1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sds1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 10
Preferred Minor : 1

    Update Time : Wed Feb  9 18:45:59 2011
          State : clean
 Active Devices : 10
Working Devices : 10
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 27aa222b - correct
         Events : 147221

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     3      65       33        3      active sync   /dev/sds1

   0     0       8      193        0      active sync   /dev/sdm1
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3      65       33        3      active sync   /dev/sds1
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1
/dev/sdt1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 0959650a:8a397113:e0647a97:fc2066cc (local to host raid)
  Creation Time : Sat Sep 25 13:23:40 2010
     Raid Level : raid6
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
     Array Size : 15628107776 (14904.13 GiB 16003.18 GB)
   Raid Devices : 10
  Total Devices : 8
Preferred Minor : 1

    Update Time : Wed Feb  9 19:20:33 2011
          State : clean
 Active Devices : 8
Working Devices : 8
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 27aa292e - correct
         Events : 147225

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     1      65       49        1      active sync   /dev/sdt1

   0     0       0        0        0      removed
   1     1      65       49        1      active sync   /dev/sdt1
   2     2      65       17        2      active sync   /dev/sdr1
   3     3       0        0        3      faulty removed
   4     4      65        1        4      active sync   /dev/sdq1
   5     5       8      161        5      active sync   /dev/sdk1
   6     6       8       81        6      active sync   /dev/sdf1
   7     7       8       97        7      active sync   /dev/sdg1
   8     8       8      177        8      active sync   /dev/sdl1
   9     9       8       65        9      active sync   /dev/sde1


On Fri, Feb 11, 2011 at 8:03 PM, NeilBrown <neilb@suse.de> wrote:
> On Fri, 11 Feb 2011 19:57:52 -0500 Christian <cjs703@gmail.com> wrote:
>
>> # mdadm -V
>> mdadm - v3.1.4 - 31st August 2010
>
> So either I didn't fix it like I thought, or I subsequently broke it.
>
> Could you send me the combined --examine output that you saved?
> That might help me reproduce the exact problem, then add it to the
> mdadm test suite.  Then fix it.
>
> Thanks,
> NeilBrown
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2011-02-12  4:08 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <AANLkTimLfJHc2waPkvjUXn=mewc37qkCd-r0GG4k=e-D@mail.gmail.com>
2011-02-11 16:48 ` RAID6 array won't start Christian
2011-02-11 20:49   ` NeilBrown
2011-02-11 21:13     ` Christian
2011-02-11 21:32       ` NeilBrown
2011-02-11 21:51         ` Christian
2011-02-11 22:36           ` Christian
2011-02-11 23:37             ` Christian
2011-02-12  0:52             ` NeilBrown
2011-02-12  0:57               ` Christian
2011-02-12  1:03                 ` NeilBrown
2011-02-12  4:08                   ` Christian

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox