From: Fabian Fischer <raid@fabianfischer.org>
To: linux-raid@vger.kernel.org
Subject: degraded raid array with bad blocks
Date: Thu, 16 Jul 2015 20:14:21 +0200 [thread overview]
Message-ID: <55A7F47D.1020004@fabianfischer.org> (raw)
[-- Attachment #1: Type: text/plain, Size: 2517 bytes --]
Hi,
today I had some problems with my mdadm raid5 (4disks). Firstly I try to
explaine what happened and what the result is:
One disk in my array has some bad blocks. After some hardware-changes
one of the intact disks was thrown out of the array due to a faulty
sata-cable.
I shut down the server and replaced the cable.
After booting, the removed disk wasn't re added to the array (maybe
because of different event count). --re-add doesn't work.
So I used --add.
Because of the bad blocks on one of the remaining disks, the rebuild
stops when reaching the first bad block. The re added disk is declared
as spare, 2 disks active and the disk with bad blocks as faulty.
/dev/md127:
Version : 1.2
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 1953512960 (1863.02 GiB 2000.40 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Thu Jul 16 19:02:09 2015
State : clean, FAILED
Active Devices : 2
Working Devices : 3
Failed Devices : 1
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Name : FiFa-Server:0
UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Events : 107223
Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 80 1 active sync /dev/sdf
5 8 32 2 active sync /dev/sdc
6 0 0 6 removed
4 8 96 - faulty /dev/sdg
6 8 64 - spare /dev/sde
In my opinion there a 3 possibilities to get the array back working. I
am not sure whether both possibilities really exist and which one is the
most promising.
- Using the 'spare'-disk as active disk. The data on the disk
should be still there.
- Ignoring the bad blocks and loose information stored in this
blocks
- force start the array without the 'spare' disk and copy the
data to backup-storage, or does the bad block will cause the
array to fail when reaching a bad block?
In the attachment you can find the output of --examine.
In can not explain why 3 disk have a Bad Block Log. According to
smart-values only sdg has Reallocated_Sector_Ct >0
Another thing I can't explain is why sdg (which is the disk with known
bad blocks) has a lower event count.
I hope I can get some great ideas how to fix my array.
Fabian
[-- Attachment #2: examine.txt --]
[-- Type: text/plain, Size: 3741 bytes --]
/dev/sdc:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : 97ccd551:8820c0e3:4ab3d67d:908a2fd9
Update Time : Thu Jul 16 19:02:09 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 9d018eea - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x8
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : 6db7566d:3c709370:12634f66:f6bfd4f6
Update Time : Thu Jul 16 19:02:09 2015
Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
Checksum : 4b73aeab - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : spare
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1968 sectors, after=1200 sectors
State : clean
Device UUID : 27df429f:c9661838:3a8024e4:d55055e8
Update Time : Thu Jul 16 19:02:09 2015
Checksum : b4327aa4 - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x8
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : b6786dff:c6be0236:10d8f95f:63afad93
Update Time : Thu Jul 16 19:02:03 2015
Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
Checksum : 397b22c8 - correct
Events : 107192
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
next reply other threads:[~2015-07-16 18:14 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-16 18:14 Fabian Fischer [this message]
2015-07-17 2:09 ` degraded raid array with bad blocks Roman Mamedov
2015-07-21 22:48 ` NeilBrown
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55A7F47D.1020004@fabianfischer.org \
--to=raid@fabianfischer.org \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).