From: Fabian Fischer <raid@fabianfischer.org>
To: linux-raid@vger.kernel.org
Subject: degraded raid array with bad blocks
Date: Thu, 16 Jul 2015 20:14:21 +0200 [thread overview]
Message-ID: <55A7F47D.1020004@fabianfischer.org> (raw)
[-- Attachment #1: Type: text/plain, Size: 2517 bytes --]
Hi,
today I had some problems with my mdadm raid5 (4disks). Firstly I try to
explaine what happened and what the result is:
One disk in my array has some bad blocks. After some hardware-changes
one of the intact disks was thrown out of the array due to a faulty
sata-cable.
I shut down the server and replaced the cable.
After booting, the removed disk wasn't re added to the array (maybe
because of different event count). --re-add doesn't work.
So I used --add.
Because of the bad blocks on one of the remaining disks, the rebuild
stops when reaching the first bad block. The re added disk is declared
as spare, 2 disks active and the disk with bad blocks as faulty.
/dev/md127:
Version : 1.2
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 1953512960 (1863.02 GiB 2000.40 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Thu Jul 16 19:02:09 2015
State : clean, FAILED
Active Devices : 2
Working Devices : 3
Failed Devices : 1
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Name : FiFa-Server:0
UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Events : 107223
Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 80 1 active sync /dev/sdf
5 8 32 2 active sync /dev/sdc
6 0 0 6 removed
4 8 96 - faulty /dev/sdg
6 8 64 - spare /dev/sde
In my opinion there a 3 possibilities to get the array back working. I
am not sure whether both possibilities really exist and which one is the
most promising.
- Using the 'spare'-disk as active disk. The data on the disk
should be still there.
- Ignoring the bad blocks and loose information stored in this
blocks
- force start the array without the 'spare' disk and copy the
data to backup-storage, or does the bad block will cause the
array to fail when reaching a bad block?
In the attachment you can find the output of --examine.
In can not explain why 3 disk have a Bad Block Log. According to
smart-values only sdg has Reallocated_Sector_Ct >0
Another thing I can't explain is why sdg (which is the disk with known
bad blocks) has a lower event count.
I hope I can get some great ideas how to fix my array.
Fabian
[-- Attachment #2: examine.txt --]
[-- Type: text/plain, Size: 3741 bytes --]
/dev/sdc:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : 97ccd551:8820c0e3:4ab3d67d:908a2fd9
Update Time : Thu Jul 16 19:02:09 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 9d018eea - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x8
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : 6db7566d:3c709370:12634f66:f6bfd4f6
Update Time : Thu Jul 16 19:02:09 2015
Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
Checksum : 4b73aeab - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : spare
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1968 sectors, after=1200 sectors
State : clean
Device UUID : 27df429f:c9661838:3a8024e4:d55055e8
Update Time : Thu Jul 16 19:02:09 2015
Checksum : b4327aa4 - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x8
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : b6786dff:c6be0236:10d8f95f:63afad93
Update Time : Thu Jul 16 19:02:03 2015
Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
Checksum : 397b22c8 - correct
Events : 107192
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
next reply other threads:[~2015-07-16 18:14 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-16 18:14 Fabian Fischer [this message]
2015-07-17 2:09 ` degraded raid array with bad blocks Roman Mamedov
2015-07-21 22:48 ` NeilBrown
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55A7F47D.1020004@fabianfischer.org \
--to=raid@fabianfischer.org \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.