All of lore.kernel.org
 help / color / mirror / Atom feed
* Recovering RAID5 with 2, actually 1, faulty disks.
@ 2015-11-23 15:28 Semyon Enskiy
  2015-11-23 15:35 ` Semyon Enskiy
  2015-11-23 18:24 ` Phil Turmel
  0 siblings, 2 replies; 17+ messages in thread
From: Semyon Enskiy @ 2015-11-23 15:28 UTC (permalink / raw)
  To: linux-raid

Hi all,

I am sorry for bad English, it is not my primary language.

Does solution, described here -
"http://marc.info/?l=linux-raid&m=144659416216285&w=2", matches for mine
issue?

Please, read issue description below, I am ready to provide additional info.

RAID1 (md1; boot), RAID10 (md2; swap, root) and
RAID5 (md3; LVM -> ext4 -> data)
are placed on 10 disks, each of them is 4TB WD Red with 3 partitions on it.
All disks model is WDC WD40EFRX-68WT0N0.
sd?1 partitions belongs to md1, sd?2 belongs to md2, sd?3 belongs to md3.

sdj caught IO error and was reinitialized under new minor and in-/dev/ IDs,
in fact partitions on it was dropped from arrays.

sdk1 and sdk2, previous sdj1 and sdj2 accordingly, was readded to their
arrays by me, but md3 stay leave sdj3 as member of array, I can't do
anything with it, only receiving errors like that:
    # mdadm /dev/md3 -r detached
    mdadm: Cannot find 8:147: No such file or directory
so sdk3 can't be readded.

So I've managed to grow md3 array with command like following, do not
remember why and strictly command:
    # mdadm --grow --raid-devices=11 -add /dev/md3 /dev/sdk3
array start to reshaping, but caught second error and reshaping process
stalled with 1536 bytes transferred. Then was executed near two useless
commands, like making sdk3 spare device.

Now md3 consists 11 devices, was 10, has 2 faulty devices, actually 1 - old
sdj3 and sdj3 under new ID - sdk3, which is slightly corrupted by writing
1536 bytes.

Seems, that I should and write data back to source block device to do it
consistent and able to reboot, recreate array with one corrupted device and
resync it.

Which data was transferred at reshaping - raw data from first block device
in the array, including mdadm metadata (/dev/sd?3), or data from array's
summary block device (/dev/md3)?

After this all happened, host was not rebooted, stay online, FS on md3 array
unavailable, smartctl reports (-H argument), that all disks are healthy.



# uname -sr
Linux 3.13.0-52-generic



# strace -p `pgrep -o mdadm` -f
Process 1250 attached
select(5, NULL, NULL, [4], {181, 601546}



# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md2 : active raid10 sdk2[12] sdg2[1] sdh2[10] sdi2[5] sdf2[9] sde2[8] sdd2[6] sdc2[4] sdb2[11] sda2[0]
      35653120 blocks super 1.2 512K chunks 2 near-copies [10/10] [UUUUUUUUUU]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md1 : active raid1 sdk1[9] sdg1[6] sdh1[7] sdi1[8] sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1] sda1[0]
      203712 blocks [10/10] [UUUUUUUUUU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md3 : active raid5 sdk3[14](S) sdg3[6] sdh3[11] sdj3[13](F) sdi3[10] sde3[4] sdf3[5] sdd3[3] sdc3[2] sdb3[12] sda3[0]
      35095924224 blocks super 1.2 level 5, 512k chunk, algorithm 2 [11/10] [UUUUUUUU_U_]
      [>....................] reshape = 0.0% (1536/3899547136) finish=10215829552.5min speed=0K/sec
      bitmap: 0/15 pages [0KB], 131072KB chunk

unused devices: <none>



# blkid /dev/sda3 /dev/sdk3
/dev/sda3: UUID="9bdc0939-4838-89af-b67c-199b2aaecb4e" UUID_SUB="fa7c541d-a25b-abd8-14ff-e1c0c6c1578c" LABEL="ubuntu:3" TYPE="linux_raid_member" 
/dev/sdk3: UUID="9bdc0939-4838-89af-b67c-199b2aaecb4e" UUID_SUB="c0155214-a9c8-e378-d368-b6f6735048a8" LABEL="ubuntu:3" TYPE="linux_raid_member"



# mdadm --detail /dev/md3
/dev/md3:
        Version : 1.2
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
     Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
  Used Dev Size : 3899547136 (3718.90 GiB 3993.14 GB)
   Raid Devices : 11
  Total Devices : 11
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Fri Nov 6 16:53:06 2015
          State : active, FAILED, reshaping 
 Active Devices : 9
Working Devices : 10
 Failed Devices : 1
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 512K

 Reshape Status : 0% complete
  Delta Devices : 1, (10->11)

           Name : ubuntu:3
           UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
         Events : 21586

    Number Major Minor RaidDevice State
       0 8 3 0 active sync /dev/sda3
      12 8 19 1 active sync /dev/sdb3
       2 8 35 2 active sync /dev/sdc3
       3 8 51 3 active sync /dev/sdd3
       4 8 67 4 active sync /dev/sde3
       5 8 83 5 active sync /dev/sdf3
       6 8 99 6 active sync /dev/sdg3
      11 8 115 7 active sync /dev/sdh3
      16 0 0 16 removed
      10 8 131 9 active sync /dev/sdi3
      20 0 0 20 removed

      13 8 147 - faulty
      14 8 163 - spare /dev/sdk3



# "mdadm --examine" output
/dev/sda3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : fa7c541d:a25babd8:14ffe1c0:c6c1578c

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
       Checksum : ac275763 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdb3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 70afdaad:b0979fb1:1b99ef18:dd58cc3a

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 1c2bf759 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdc3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 345b0212:6c3a0ae8:d8ed9273:ecc53573

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
       Checksum : 49c30753 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 2
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdd3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : a4b8b425:b0307fe5:9eb8e3ac:ae985311

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
       Checksum : 3258f890 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 3
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sde3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 4490843e:d2c5c2e7:d222fa0a:f98b277c

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
       Checksum : 1656c2d2 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 4
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdf3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 1d41056f:88149b1a:48e939e5:8c775e23

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
       Checksum : fb26746a - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 5
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdg3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 375d85e2:9077dd68:bc1327af:8fee4112

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
       Checksum : 75b99505 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 6
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdh3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 67cd2bfa:affa578c:e0c1286a:bd4d2627

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 80c895f3 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 7
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdi3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : a5f2cea8:a89c88a0:0dd3900c:624f0491

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
       Checksum : 4fda6fb3 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 9
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdj3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : c0155214:a9c8e378:d368b6f6:735048a8

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 952a55f2 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : spare
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



/dev/sdk3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x45
     Array UUID : 9bdc0939:483889af:b67c199b:2aaecb4e
           Name : ubuntu:3
  Creation Time : Thu Jul 24 16:06:32 2014
     Raid Level : raid5
   Raid Devices : 11

 Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
     Array Size : 38995471360 (37188.98 GiB 39931.36 GB)
  Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
    Data Offset : 262144 sectors
     New Offset : 251904 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : c0155214:a9c8e378:d368b6f6:735048a8

Internal Bitmap : 8 sectors from superblock
  Reshape pos'n : 0
  Delta Devices : 1 (10->11)

    Update Time : Fri Nov 6 16:53:06 2015
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 952a55f2 - correct
         Events : 21586

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : spare
   Array State : AAAAAAAA.A. ('A' == active, '.' == missing, 'R' == replacing)



# grep'ed smartctl output #1, 
/dev/sda
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
/dev/sdb
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
/dev/sdc
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
/dev/sdd
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
/dev/sde
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
/dev/sdf
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
/dev/sdg
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
/dev/sdh
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
/dev/sdi
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
/dev/sdk
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)



# grep 'sd.3' /proc/partitions
   8 3 3899678535 sda3
   8 19 3899678535 sdb3
   8 35 3899678535 sdc3
   8 51 3899678535 sdd3
   8 67 3899678535 sde3
   8 83 3899678535 sdf3
   8 99 3899678535 sdg3
   8 115 3899678535 sdh3
   8 131 3899678535 sdi3
   8 163 3899678535 sdk3



# grep'ed smartctl output #2
/dev/sda
  9 Power_On_Hours -O--CK 082 082 000 - 13233
/dev/sdb
  9 Power_On_Hours -O--CK 085 085 000 - 11051
/dev/sdc
  9 Power_On_Hours -O--CK 085 085 000 - 11379
/dev/sdd
  9 Power_On_Hours -O--CK 085 085 000 - 11403
/dev/sde
  9 Power_On_Hours -O--CK 081 081 000 - 14233
/dev/sdf
  9 Power_On_Hours -O--CK 085 085 000 - 11357
/dev/sdg
  9 Power_On_Hours -O--CK 085 085 000 - 11382
/dev/sdh
  9 Power_On_Hours -O--CK 085 085 000 - 11097
/dev/sdi
  9 Power_On_Hours -O--CK 082 082 000 - 13256
/dev/sdk
  9 Power_On_Hours -O--CK 089 089 000 - 8413



# dmesg
"http://paste.debian.net/335374/"
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 17+ messages in thread
* Re: Recovering RAID5 with 2, actually 1, faulty disks.
@ 2015-12-02 13:35 Semyon Enskiy
  0 siblings, 0 replies; 17+ messages in thread
From: Semyon Enskiy @ 2015-12-02 13:35 UTC (permalink / raw)
  To: philip, linux-raid@vger.kernel.org

Hello, Phil, thank you for help.

Writing to report, that array and LVM/ext4 on it successfully restored.



# Before rebooting and committing changes, dump of first 10 MB was done, 
now restore headers.
# for i in f i h g d e b c a j; do dd if=devsd"$i"3-head-bigger.raw 
of=/dev/sd"$i"3; done

# Without revert-reshape assembling.
# mdadm --create --assume-clean --level=5 --raid-devices=10 
--metadata=1.2 /dev/md3 /dev/sd{f,i,h,g,d,e,b,c,a,j}3

# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] 
[raid4] [raid10]
md3 : active raid5 sdj3[9] sda3[8] sdc3[7] sdb3[6] sde3[5] sdd3[4] 
sdg3[3] sdh3[2] sdi3[1] sdf3[0]
       35095924224 blocks super 1.2 level 5, 512k chunk, algorithm 2 
[10/10] [UUUUUUUUUU]
       bitmap: 1/30 pages [4KB], 65536KB chunk

# blkid /dev/md3
/dev/md3: UUID="4bz7kl-2QWk-4YZX-IDmH-IFXL-eM3l-F2dB2n" TYPE="LVM2_member"

# vgchange -aay

# blkid /dev/vg/data
/dev/vg/data: UUID="60e11554-074d-4156-a4d3-ed43318f5446" TYPE="ext4"

# fsck.ext4 /dev/vg/data
e2fsck 1.42.9 (4-Feb-2014)
/dev/vg/data: clean, 464/548112384 files, 4798924497/8769786880 blocks

# mount /dev/vg/data

# echo check > /sys/block/md3/md/sync_action



# mdadm --detail /dev/md3
/dev/md3:
         Version : 1.2
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 3899547136 (3718.90 GiB 3993.14 GB)
    Raid Devices : 10
   Total Devices : 10
     Persistence : Superblock is persistent

   Intent Bitmap : Internal

     Update Time : Wed Dec  2 13:40:04 2015
           State : active
  Active Devices : 10
Working Devices : 10
  Failed Devices : 0
   Spare Devices : 0

          Layout : left-symmetric
      Chunk Size : 512K

            Name : ubuntu:3  (local to host ubuntu)
            UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
          Events : 2

     Number   Major   Minor   RaidDevice State
        0       8       83        0      active sync   /dev/sdf3
        1       8      131        1      active sync   /dev/sdi3
        2       8      115        2      active sync   /dev/sdh3
        3       8       99        3      active sync   /dev/sdg3
        4       8       51        4      active sync   /dev/sdd3
        5       8       67        5      active sync   /dev/sde3
        6       8       19        6      active sync   /dev/sdb3
        7       8       35        7      active sync   /dev/sdc3
        8       8        3        8      active sync   /dev/sda3
        9       8      147        9      active sync   /dev/sdj3



# mdadm --examine /dev/sd?3
/dev/sda3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : 074ebbe7:81662e37:cc777096:3d50d481

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : d47becef - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 8
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sdb3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : 4adc3cf2:9d813d1e:ab8044ab:0cc5afe8

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 41bc13fb - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 6
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sdc3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : ddc9c73b:007f6c0f:35a03e5c:e5fc8157

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 9c425653 - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 7
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sdd3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : 73077551:029e51dc:6d036bf4:b81195a4

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 64142af5 - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 4
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sde3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : 1edcf3a7:9decc271:262271a6:54cff640

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 9e6c2a90 - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 5
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sdf3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : 0438b536:b5943d93:70280dd9:70952d14

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 547afaef - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 0
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sdg3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : e16e0db6:5d8e9016:84d2034b:7519f24d

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 2e15990 - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 3
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sdh3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : 32bdd4dd:934c3ba5:adc8bfc2:0f1f323e

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 214f61da - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 2
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sdi3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : bd0d56ba:78e5ee76:d600d587:ab65cbc9

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 2032ca0e - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 1
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)
/dev/sdj3:
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x1
      Array UUID : 39b91f6d:7fd12764:825bb342:dd4d14fd
            Name : ubuntu:3  (local to host ubuntu)
   Creation Time : Wed Dec  2 13:09:01 2015
      Raid Level : raid5
    Raid Devices : 10

  Avail Dev Size : 7799094927 (3718.90 GiB 3993.14 GB)
      Array Size : 35095924224 (33470.08 GiB 35938.23 GB)
   Used Dev Size : 7799094272 (3718.90 GiB 3993.14 GB)
     Data Offset : 262144 sectors
    Super Offset : 8 sectors
    Unused Space : before=262056 sectors, after=655 sectors
           State : clean
     Device UUID : 35d4e323:8e2485cb:2c6492a1:e9f8e215

Internal Bitmap : 8 sectors from superblock
     Update Time : Wed Dec  2 15:19:29 2015
   Bad Block Log : 512 entries available at offset 72 sectors
        Checksum : 442bc637 - correct
          Events : 2

          Layout : left-symmetric
      Chunk Size : 512K

    Device Role : Active device 9
    Array State : AAAAAAAAAA ('A' == active, '.' == missing, 'R' == 
replacing)

^ permalink raw reply	[flat|nested] 17+ messages in thread
end of thread, other threads:[~2015-12-02 13:35 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-11-23 15:28 Recovering RAID5 with 2, actually 1, faulty disks Semyon Enskiy
2015-11-23 15:35 ` Semyon Enskiy
2015-11-23 15:47   ` Wols Lists
2015-11-23 15:57     ` Semyon Enskiy
2015-11-23 16:08       ` Wols Lists
2015-11-23 18:24 ` Phil Turmel
2015-11-25 12:12   ` Semyon Enskiy
2015-11-25 13:22     ` Phil Turmel
2015-11-25 14:48       ` Semyon Enskiy
2015-11-25 14:55         ` Semyon Enskiy
2015-11-25 15:44           ` Phil Turmel
2015-11-25 16:29             ` Semyon Enskiy
2015-11-25 16:58               ` Phil Turmel
2015-11-25 17:32                 ` Semyon Enskiy
2015-11-25 17:49                   ` Phil Turmel
2015-11-26 13:21                     ` Semyon Enskiy
  -- strict thread matches above, loose matches on Subject: below --
2015-12-02 13:35 Semyon Enskiy

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.